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The D.C. School Choice Incentive 
Act created the first private 
kindergarten-through-grade- 1 2 
school-choice program supported 
by federal funds. The program was 
named the D.C. Opportunity 
Scholarship Program (OSP). GAO 
was asked to assess the (1) 
accountability mechanisms 
governing the use of funds, (2) 
results of the grantee’s efforts to 
meet the program’s recruiting 
priorities and eligibility 
requirements and inform parents of 
their choices, and (3) extent that 
the evaluation of OSP reflects 
statutory requirements and the 
implementation of the program 
supported the detection of useful 
and generalizable findings. To 
assess the program, GAO analyzed 
financial, program, and evaluation 
data. GAO did not assess the 
performance of participating 
private schools nor did GAO 
evaluate the impact of the program. 



What GAO Recommends 



The Secretary of Education should 
direct the grantee to improve 
internal controls, continue to 
integrate its financial systems, 
improve monitoring, and provide 
accurate information to parents. 

The Secretary and the Mayor of the 
District of Columbia should ensure 
participating schools are in 
compliance with District 
requirements. The Secretary 
should ensure that programs are 
implemented in accordance with 
any required evaluation. The 
entities providing comments on 
this report generally agreed with 
our recommendations but 
disagreed with many findings. 

To view the full product, including the scope 
and methodology, click on GAO-08-9. 

For more information, contact Cornelia M. 
Ashby at 202-512-7215 or ashbyc@gao.gov. 
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What GAO Found 

Following the Department of Education’s selection of the Washington 
Scholarship Fund (WSF) to operate the District of Columbia Opportunity 
Scholarship Program, WSF greatly expanded its operations from $150,000 in 
federal and foundation grants in fiscal year 2004 to $12.9 million in 2006 
without sufficient accountability mechanisms to govern the use of the funds. 
With such rapid expansion, WSF had little time to design and implement the 
needed systems, procedmes, and internal controls for managing such a major 
increase hr its operations. WSF’s accountability was further weakened by high 
staff turnover, a lack of detailed fiscal policies, and nonintegrated accounting 
functions. We found that WSF did not adhere to its own procedures for 
making scholarship payments, and WSF’s use of OSP funds to pay tuition for 
students attending schools that do not normally charge students tuition is not 
in accordance with the Act. Additionally, on the basis of a lack of 
documentation about whether before-and-after care included academic 
support, GAO was unable to determine if use of OSP fimds to pay these fees 
was in accordance with the Act. 

Despite recruitment efforts and efforts to inform parents of their choices, 

WSF faced challenges recruiting students from schools designated as in need 
of improvement, ensuring private school quality and placement opportunities, 
and providing parents with accurate information regarding private schools. 
Students who were offered scholarships generally reported income consistent 
with OSP’s financial eligibility standards, but, among students offered 
scholarships, students from schools in need of improvement were 
underrepresented relative to their presence in the population of District 
students. Although most private schools in the District officially participated 
in the program, the schools varied widely in the number of openings available 
to scholarship students, and few openings were available at the secondary 
level. The characteristics of participating schools varied, and some schools 
did not meet basic requirements to operate in the District, but the information 
WSF provided to parents to help them choose schools for children was not 
always complete and correct. 

The evaluation contractor developed a strong evaluation design that reflected 
the statutory requirements and used random assignment to strengthen 
comparisons between students offered and not offered scholarships. 

However, factors related to program implementation limited the ability to 
perform comparisons directed by the Act and the usefulness and 
generalizability of findings. For example, the Act directed the evaluation 
contractor to use the same test as the District used to measure achievement 
and to compare the achievement of students offered scholarships with 
students in District public schools. The contractor did use the test used by the 
District, but District officials adopted a new testing program the second year 
of the evaluation, making it infeasible to compare students offered 
scholarships with students in District public schools. 
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Accountability * Integrity * Reliability 



United States Government Acconntability Office 
Washington, DC 20548 



November 1, 2007 

The Honorable Edward M. Kennedy 

Chairman, Committee on Health, Education, Labor, and Pensions 
United States Senate 

The Honorable Richard J. Durbin 

Chairman, Subcommittee on Financial Services and General Government 
Committee on Appropriations 
United States Senate 

The Honorable Eleanor Holmes Norton 
House of Representatives 

In an effort to enhance educational opportimity for children from low- 
income families in the District of Colmnbia (the District), Congress passed 
the D.C. School Choice Incentive Act in January 2004) The Act created a 
private school scholarship program, subsequently called the D.C. 
Opportunity Scholarship Program (OSP), for low-income students living in 
the District of Columbia, setting a precedent for the use of federal funds to 
provide scholarships to finance private education for students in 
kindergarten through grade 12. The purpose of the Act is to provide low- 
income parents of students in the District, particularly parents of students 
who attend public schools identified as in need of improvement imder the 
No Child Left Behind Act,^ “with expanded opportunities for enrolling their 
children in higher performing schools in the District” by providing annual 
scholarships of up to $7,600 per child. The D.C. School Choice Incentive 
Act directed the Secretary of Education to award a grant for up to 6 yecirs 
to an eligible entity or entities to operate the program, giving priority to 
grant applicants who would most effectively give priority to students from 
schools in need of improvement, target resources to students and families 
that lack financial resources to take advantage of available educational 
options, and provide students and families with the widest range of 
educational options. The Act also directed the Secretary and the Mayor of 
the District of Columbia to select an independent evaluator to compare 
outcomes, including academic achievement, for several groups of children. 



' Public Law No. 108-199, title 111, 1118 Stat. 3, 126 (Jan. 23, 2004). 

^The No Child Left Behind Act of 2001 reauthorized the Elementary and Secondary 
Education Act. 



Page 1 



GAO-08-9 District of Columbia Opportunity Scholarship Program 



such as participating eligible students and eligible students who were not 
offered scholarships and participating eligible students and students in 
District Public Schools. The evaluation was also to examine the effect of 
the program on District public schools. 

You asked us to assess the implementation of the program, including the 

(1) accountability mechanisms in place governing the use of funds, 

(2) results of the grantee’s efforts to meet the program’s recruiting 
priorities and eligibility requirements and inform parents about their 
choices, and (3) extent to which the evaluation reflects statutory 
requirements and the implementation of the program supported the 
detection of useful and generalizable findings. 

To address these issues, we used multiple strategies. To assess the 
accountability mechanisms governing the use of funds, we interviewed 
officials of the Washington Scholarship Fund (WSF), the grantee operating 
the program, of the Department of Education’s Office of Innovation and 
Improvement and of the District of Columbia; reviewed documents 
describing the grantee’s processes for implementing the program; analyzed 
WSF’s financial statements; obtained and reviewed detailed financial data 
from WSF; and tested specific internal controls over payments. To assess 
the results of the grantee’s efforts to meet the program’s recruiting 
priorities and eligibility requirements and inform parents about their 
choices, we interviewed officials at the Office of Innovation and 
Improvement and the grantee; examined pertinent documentation; and 
analyzed students’ demographic data, characteristics of participating 
schools, and information provided to parents. We reviewed information on 
private schools from WSF and public sources, but we did not assess 
specific private schools or their performance. We did not evaluate the 
impact of the program, which was the subject of Education’s evaluation. 
We reviewed WSF’s process for collecting financial eligibility information, 
the records maintained by WSF on household income data reported by 
parents of scholarship recipients, and the findings on eligibility made by 
the grantee’s auditor. We did not perform independent testing of eligibility. 
To assess the extent to which the evaluation reflects statutory 
requirements and program implementation supported the detection of 
useful and generalizable findings, we interviewed officials at Education’s 
Institute of Education Sciences and at Westat — the evaluation 
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contractor — reviewed Westat’s and its affiliates’^ reports for year one and 
year two of the OSP that described the participation of students and 
schools in the program, analysis plan, and evaluation of the results of the 
program after 1 year of participation. In assessing data reliability, we 
found problems with WSF’s and the evaluation contractor’s program data 
systems, which we discuss in appendix 1, that limited the analyses that we 
could perform. We brought these issues to the attention of Education, 
WSF, and the evaluation contractor. However, we were able to perform 
sufficient analyses to answer the study’s questions. We conducted our 
work between September 2006 and September 2007 in accordance with 
generally accepted government auditing standards. (For more detail on 
our scope and methodology, see app. 1.) 



Results in Brief 



Following the Department of Education’s selection of WSF to operate the 
District of Columbia Opportunity Scholarship Program, WSF greatly 
expanded its operations from $160,000 in federal and foundation grants in 
fiscal year 2004 to $12.9 million in 2006 without sufficient accountability 
mechanisms to govern the use of the OSP funds. Rapid expansion and new 
demands put at risk WSF’s accountability for its use and safeguarding of 
federal funds. This accountability risk was increased by high turnover in 
key staff positions. WSF did not develop the comprehensive and detailed 
fiscal policies and procedures needed for internal control over its financial 
and programmatic operations. WSF’s draft policies and procedures lacked 
key control activities and did not detail procedures for approving 
scholarship payments or require documentation that procedures were 
carried out as required. WSF’s automated financial systems were not 
interfaced, and reliance on manual processes between automated systems 
increased the risk of error. Serious weaknesses in the implementation of 
internal controls affected the processing of OSP scholarship payments, 
reconciliations between financial records and bank statements, and 
assurance that participating schools were financially responsible. For 
example, while WSF established policies to help ensure that OSP funds are 
used to pay for services in accordance with the Act establishing the 
program, WSF’s use of OSP funds to pay tuition for students attending 
schools that do not normally charge students tuition is not in accordance 
with the Act. However, WSF maintains that students attending these 



^The participation reports and analysis plan were prepared by Westat in affiliation with 
Georgetown University and Chesapeake Research Associates; the program evaluation was 
prepared by Westat in affiliation with the University of Arkansas, Georgetown University, 
and Chesapeake Research Associates. 
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schools are supported by scholarship dollars just as those who receive 
OSP funds. Furthermore, our review questions certain other payments, 
but we were unable to determine if they were allowable because these 
payments lacked documentation. 

Despite efforts to recruit eligible applicants and private schools and 
inform parents, WSF faced challenges recruiting students from schools in 
need of improvement, ensuring private school placement opportunities 
and quality, and providing parents with accurate information regarding 
private schools. WSF designed procedures to ensure that scholarship 
recipients met program financial eligibility standards and income data 
reported by parents of participants were broadly consistent with income 
guidelines. While students from schools in need of improvement had 
priority for receiving scholarships, for each year the program has 
operated, the percentage of students from schools in need of improvement 
who were offered scholarships has been smaller than the percentage of 
students in District public schools who attended such schools. Although 
most private schools in the District officially participated in the program, 
the schools varied widely in the number of openings available to 
scholarship students, and very few openings were available at the 
secondary level. In school year 2006-2007, more than half the scholarship 
students were concentrated in 19 participating private schools, and 16 
private schools had enrolled five or fewer OSP students. The extent to 
which private schools had characteristics associated with high-quality 
educational programs has varied. For example, some schools reported that 
some of their teachers lacked at least a bachelor’s degree. Furthermore, 
some participating schools did not meet basic requirements to operate in 
the District. For example, a few had no certificate of occupancy on file 
with the District or had certificates that did not specify educational use. 
Despite important variation among schools, the grantee did not always 
provide parents with complete and accurate information on their 
characteristics. For example, WSF provided parents inaccurate 
information on teachers’ qualifications and tuition for some schools. 

The evaluation contractor developed a strong methodology that used 
random assignment to strengthen comparisons between students offered 
and not offered scholarships, but factors related to program 
implementation limited the ability to perform the comparisons directed by 
the Act and the usefulness and generalizability of evaluation findings. For 
example, although the Act directed the evaluators to compare the 
achievement of participating eligible students who used scholarships with 
students in the same grade in District public schools, the evaluation 
contractor could not reasonably or cost-effectively make these 
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comparisons. Although the evaluation contractor administered the same 
tests to measure achievement as were given by the District in the first 
year, the District adopted a new achievement testing program during the 
second year of the program for students in grades 3-8 and grade 10. This 
meant that a common test was no longer available for the program’s 5 
years of operation and made it necessary to pay the evaluation contractor 
to test more students than planned because, in addition to testing students 
with OSP scholarships in private schools, the contractor also had to test 
eligible applicants in District public schools who were not offered 
scholarships. Also, the Act called for comparisons of dropout rates, 
graduation rates, and college admission rates of scholarship program 
participants and nonparticipants, but the limited placements offered by 
high schools restricted the number of high school students and graduates 
in the evaluation and eliminated the evaluator’s ability to make these 
comparisons. Because the WSF awarded many scholarships nonrandomly 
and to some students who already attended private schools, the number of 
students who could be included in the evaluation was diminished, 
reducing the ability of the evaluation’s statistical tests to identify 
differences between groups and the strength of possible findings. Other 
factors arising from the operation of the program, including the inability of 
some students who were offered scholarships to find placements and the 
provision of school or community based supplemental services to some 
scholarship recipients might have made the interpretation of findings 
difficult and also might have limited the usefulness of the evaluation to 
programs in other jurisdictions. 

To improve OSP, we are making several recommendations to the 
Secretary of Education and the Mayor of the District of Columbia. To 
better focus resources on achieving the program’s goals of providing 
accountability, quality educational experiences, and allowing parents to 
make an informed school choice for their child, we recommend that the 
Secretary direct the grantee to establish policies and procedures that 
improve certain internal controls, continue efforts to integrate its financial 
systems, improve monitoring, and provide accurate and complete 
information to parents. We are also recommending that the Secretary and 
the Mayor of the District of Columbia take specific steps to improve 
private school compliance with applicable District requirements and 
improve oversight of the program. Finally, we recommend that, in 
planning for future programs for which Congress has required an 
evaluation. Education should oversee the program to ensure consistency 
between program implementation and the evaluation design. 
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We sent this report to the Office of the Mayor of the District of Columbia, 
Education and WSF for review and comment. (Comments are reprinted in 
appendixes III, IV, and V, respectively.) These entities generally agreed 
with our recommendations, but Education and WSF disagreed with many 
of our findings. Education officials raised concerns related to scope of the 
assignment and stated that the draft report did not present a complete and 
balanced picture of issues such as the extent to which OSP students 
previously attended schools in need of improvement. In addition. 
Education provided comments indicating that GAO should have focused 
on the percentage of students whose former schools were identified for 
improvement by the end of the students’ first year in the OSP. We disagree. 
Scholarship lottery organizers could not be expected to give priority to 
students from schools that had not yet been identified. Moreover, 
regardless of which year is used, the percentage of students from schools 
in need of improvement was smaller among scholarship recipients than 
among the general population of District students from whom scholarship 
recipients could have been recruited. Education also indicated we should 
have provided more information on the speed with which the program was 
implemented, satisfaction among parents, demand for scholarships, and 
participating schools’ traditional independence in areas such as hiring and 
establishing teacher qualifications. As discussed further in our response 
beginning on page 46, we believe that these subjects were addressed in a 
complete and balanced fashion to the extent that they were included in the 
scope of our review. Moreover, we found that lack of reliable data on 
applicants limited the analysis we could perform to determine the demand 
for scholarships. We agree that participating schools had independence in 
areas such as hiring and establishing teacher qualifications, but note that 
participating schools are also required to be in compliance with District 
education and safety requirements. In its comments, WSF also 
acknowledged that some of GAO’s recommendations, findings, and 
observations were valid and useful and that it had taken or will take action 
to address some of them but disagreed with many of our observations, 
interpretations, and findings. While WSF implies that the deficiencies 
noted in this report are minor in nature and disagreed with our conclusion 
that the Act prohibited payments to schools that customarily charged no 
tuition, we stress that the issues we identified are indicative of potential 
problems and if not addressed could have a material, detrimental effect on 
WSF’s accountability over federal funds, especially when combined with 
the accounting systems and cash reconciliation weaknesses that existed. 
As we noted above in our response to Education, we believe the findings 
and interpretations are accurate as stated in the draft and therefore have 
not made changes other than technical corrections. 
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Background 



The D.C. School Choice The D.C. School Choice Incentive Act of 2003 directs the Secretary of 
Incentive Act Education to award a grant on a competitive basis for up to 5 years to an 

entity or entities to use to make scholarship payments to parents of 
eligible students to be used for private school tuition, fees, and 
transportation expenses. The Act directs the grantee to ensure that the 
amount of tuition or fees that a participating private school charges a 
scholarship student does not exceed the tuition or fees that the school 
customarily charges to students not participating in the scholarship 
program. Under the Act, Education could not approve a request for a grant 
unless the entity’s application detailed how it would, among other things: 

• address the priorities identified in the Act — ^that is, giving priority to 
students who attended schools identified as in need of improvement,'' 
targeting resources to families that lacked resources to take advantage 
of available educational options, and providing students and families 
with the widest range of educational choice; 

• ensure that if more eligible students seek admission in the program 
than the program can accommodate, eligible students are selected for 
admission through a random selection process that gives weight to the 
program priorities; 

• notify parents of eligible students of expanded choice opportunities 
and ensure that parents receive sufficient information about their 
options to allow the parents to make informed decisions; 

• determine the amount that would be provided to parents for the tuition, 
fees, and transportation payments, if any; 

• seek private elementary and secondary schools in the District to 
participate in the program, and ensure that participating schools would 
meet the application and reporting requirements of this Act; 

• ensure that participating schools were financially responsible and 
would use funds effectively; and 

• address the renewal of scholarships to participating eligible students, 
including continued eligibility. 



'' Under the No Child Left Behind Act, if a school that receives Title I funds fails to meet 
certain performance goals for 2 years, it is designated as in need of improvement. Schools 
that do not meet these goals in 4 years are subject to corrective action and after 5 years, are 
subject to restructuring. This report uses the term “in need of improvement” to refer to all 
three categories. 
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The Act limits payments for each scholarship student to $7,600 per year. If 
tuition exceeds the $7,600 cap, schools may supplement the $7,600 with 
private scholarships or donated funds, absorb the difference, or work out 
payment plans with parents or guardians. According to the program 
requirements established by WSF, participating private schools may apply 
their own admission standards and decide whether to admit students who 
have received an OSP scholarship. The program’s funding allowed 
scholarships for about 2,000 District students out of a public school 
population of about 72,000. (App. II presents a comparison of the District 
of Columbia school choice program and programs in several states and 
cities.) 

The Department of Education’s Office of Innovation and Improvement 
awarded a grant to operate the program to WSF, a nonprofit organization 
in the District of Columbia that had experience providing privately funded 
scholarships to low-income students. WSF offered its first scholarships 
under OSP for the 2004-2006 school year, and the program recently 
completed its third year of operation with the conclusion of the 2006-07 
school year. The Act required that scholarships be awarded through 
lotteries if the number of eligible applicants exceeded the number of 
private school placements available or the number of scholarships that the 
program could accommodate. 

In these lotteries, students were assigned different priorities for receiving 
a scholarship. Students attending schools in need of improvement 
received the highest priority, followed by students who had attended a 
public school not in need of improvement, followed by students from 
private schools. The number of District schools assigned the designation 
of needing improvement has increased from 2003 to 2006, and the 
designations are updated following the school year. 

The Act also required that the Secretary award a contract to an entity 
jointly selected by the Secretary and the Mayor of the District of Columbia 
for an independent evaluation that would use “the strongest possible 
research design” to determine the effectiveness of the program. The 
evaluators were to provide (1) a comparison of the academic achievement 
of participating eligible applicants who were offered scholarships and 
those who were not; (2) a comparison of the academic achievement of 
applicants who were offered scholarships and children attending District 
of Columbia Public Schools; (3) comparisons of drop-out rates, graduation 
rates, and college admission rates of students who participate in the 
program with students in District schools who do not participate; (4) a 
comparison of the safety of schools attended by participants in the 
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program and the schools attended by students who do not participate in 
the program; (5) measures of the effect of the program on District public 
schools and their students; and (6) the success of the program in 
expanding choice for parents. The Secretary awarded the evaluation 
contract to Westat and its affiliates. 



Additional Laws and 
Agreements Governing 
Administration of the OSP 



The Act required that Education and the District enter into a memorandum 
of understanding (MOU) regarding the implementation of the D. C. School 
Choice Incentive Program. It directed that the MOU conform to 
requirements contained in the conference report accompanying the Act. In 
that report, the conferees stated their expectation that the MOU would 
include (1) strong accountability measures and specifications for program 
performance evaluations; (2) specifications for a lottery system providing 
for fair and unbiased acceptance of students into the program; (3) 
provisions for joint oversight of the program’s operations by the Mayor of 
the District of Columbia and the Secretary of Education; (4) the 
methodology for the evaluation and selection of participating schools that 
have met the District of Columbia’s current licensure requirements; (5) the 
methodology for determining the tuition and fees of participating schools, 
including the actual cost; (6) requirements for the development of 
appropriate oversight and accountability measures; and (7) teacher quality 
criteria. 



The MOU reflected this language in part. It identified Education’s Deputy 
Under Secretary of Innovation and Improvement as having lead 
responsibility for implementation of the OSP, Education’s Institute of 
Education Sciences as having lead responsibility for implementation of the 
evaluation, and the Mayor or his designee as having lead responsibility for 
carrying out cooperative activities between the District and Education. It 
also specified that Education and the Mayor would jointly oversee the 
program and ensure it was carried out in accordance with statutory 
requirements, the approved grant application, and sound management and 
educational principles. Also, the MOU conditioned schools’ participation, 
in part, on “operating lawfully within the District of Columbia,” -meaning 
that participating schools meet all the District of Columbia’s current 
requirements. For example, the Code of District of Columbia Municipal 
Regulations requires that for purposes of the District’s compulsory 
attendance law, private schools must provide evidence that they are 
currently accredited by or undergoing the accreditation process from one 
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of seven named organizations or any other accrediting body approved by 
the Board of Education, or that they submit evidence of their educational 
soundness acceptable to the Board/ Accreditation provides assurance that 
a school’s course credits can be counted toward a District diploma or 
accepted by other schools. Operating lawfully in the District of Columbia 
also entails meeting additional District requirements, such as securing a 
certificate of occupancy to demonstrate compliance with health, safety, 
and fire code requirements. WSF’s grant application indicated that it 
would require participating private schools to “attach their Certificate of 
Occupancy [to their letter of intent to participate] and document that they 
are in compliance with all relevant D.C. health, safety, and fire code 
requirements.” Further, the MOU stated that the grantee was to provide 
information on each participating private school to parents of students 
offered scholarships including, but not limited to, teacher qualifications, 
the academic achievement of the school’s students, and the safety and 
environment of the school. However, the MOU did not include a 
methodology for determining actual tuition and fees, or specify oversight 
and accountability measures and teacher quality criteria. 



Initiation and Funding of 
the Opportunity 
Scholarship Program 



Soon after the D.C. School Choice Incentive Act was passed. Education 
awarded the grant to operate the OSP to the Washington Scholarship 
Fund, which was the only entity that submitted a complete proposal. Table 
1 shows the timing of key events in the implementation and operation of 
OSP. 



^ D.C. Municipal Regulations 2100.2 and 2100.3. 
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Table 1: Key Events in Implementing and Operating the OSP 


Date 


Event 


January 23, 2004 


Congress enacted the D.C. School Choice Incentive Act of 2003 


February 3, 2004 


The District’s Mayor selected Fight for Children to serve as the interim, nonprofit entity charged with 
conducting early outreach and publicity efforts 


March 24, 2004 


Education awarded the grant to Washington Scholarship Fund to operate the DC School Choice 
Program 


March 26, 2004 


Education awarded a contract to provide technical assistance for placement lotteries 


Late March 2004 


Outreach efforts and preliminary data on interested families were transferred from Fight for Children to 
WSF 


Late March-late August 2004 


WSF developed application and eligibility determination processes, and program management 
systems and continued multifaceted recruitment and outreach efforts 


April 16, 2004 


The evaluation contractor and its affiliates submitted a proposal to conduct the evaluation 


Mid-April -Mid-May 2004 


WSF collected its first applications from potential scholarship families and held orientation meetings at 
the Washington Convention Center 


April-May 2004 


Families submitted applications, and WSF and its contractor processed applications and conducted 
eligibility determination 


May 1 1 , 2004 


Deadline for private schools to report available number of openings by grade for first school year of 
program 


June 17, 2004 


WSF conducted lottery for awarding year one scholarships 


July 21, 2004 


Education awarded a contract for the evaluation of the program to the firm contracted to provide 
technical assistance on placement lotteries 


August/September 2004 


First school year of Opportunity Scholarship program began with 1,027 OSP students enrolled in 
private schools 


May 2005 


WSF held lotteries for awarding scholarships 


August/September 2005 


Second school year began; 1 ,661 OSP students were enrolled in private schools 


March 2006 


Deadline for parents to apply for a scholarship for the 2006-07 school year; only children entering 
grades kindergarten through 5th grade could apply for new scholarships. 


AugusfSeptember 2006 


Third school year began, with about 1,743 OSP students enrolled in private schools 


December 20, 2006 


Congress passed legislation raising the eligible income level to 300 percent of the poverty line for 
students who received scholarships during the first two years of the program 


June 2007 


Education released Evaluation of the DC Opportunity Scholarship Program: Impacts After One Year 



Source: GAO analysis of information from Education and WSF. 



As shown in table 2, $14 million was initially appropriated to Education in 
each of fiscal years 2004 through 2006. Each year’s appropriation was 
reduced through subsequent rescissions. Table 2 also shows the flow of 
funds from Education to the program. Specifically, of the funds for fiscal 
year 2006, Education received $13.86 million. Of this amount, about 
$12.46 million was awarded to WSF in the form of a federal grant to 
operate the OSP. The amount of funding retained by Education, 

$1.4 million, was provided to Education’s Office of Innovation and 
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Improvement for program administration and oversight ($0.1 million) and 
to Education’s Institute of Education Sciences for an evaluation of the 
program ($1.3 million) as mandated by the Act. Funds disbursed to the 
grantee, WSF, are to be expended for tuition, fees, transportation, and 
program administration. According to WSF, the majority of funds donated 
by private sources are used for eligibility determination, infrastructure, 
and other technical systems development and maintenance, and student 
placement. 



Table 2: Opportunity Scholarship Program Federal Funding for 


Fiscal Years 2004-2006 






Sources and disbursement of funds 


2004 


2005 


2006 


Appropriation to Education 


Federal appropriation 


$14,000,000 


$14,000,000 


$14,000,000 


Rescission" 


(82,600) 


(112,000) 


(140,000) 


Net appropriations to the U.S. Department of Education 


13,917,400 


13,888,000 


13,860,000 


Disbursement of grant funds 


Department of Education 


Institute of Education Sciences 


1 ,344,822 


1 ,308,640 


1 ,305,877 


Office of Innovation and Improvement (Oil) 


66,800 


100,000 


99,923 


Grant provided to WSF 


12,505,778 


12,479,360 


12,454,200 


Total funds disbursed to Education and WSF 


13,917,400 


13,888,000 


13,860,000 



Source: GAO analysis of appropriations laws and allotments data. 



"Congress enacted across-the-board rescissions for appropriations made for fiscai years 2004, 2005, 
and 2006. The rescissions were 0.59 percent for fiscai year 2004, 0.80 percent for fiscai year 2005, 
and 1 percent for fiscai year 2006. 



The Act provided that the grantee may not use more than 3 percent of the 
total amount provided under the grant each year for the administrative 
expenses of carrying out the program during the year, including (1) 
determining the eligibility of students to participate, (2) providing 
information about the program and the schools involved to parents of 
eligible students, (3) selecting students to receive scholarships, (4) 
determining the amount of scholarships and issuing the scholarships to 
eligible students, (5) compiling and maintaining financial and 
programmatic records, and (6) providing funds to assist parents in meeting 
expenses that might otherwise preclude the participation of their child in 
the program. WSF officials defined administrative expenses as any 
program expenses other than tuition, fees, and transportation. These 
administrative expenses generally included direct expenses, such as 
student eligibility reviews and payroll, and indirect expenses, such as rent 
and utilities. 
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WSF’s Accountability 
and Internal Control 
Were Inadequate for 
OSP, and Some 
Payments Were Not 
Made in Accordance 
with the Act While 
Others Raise 
Questions about 
Compliance 



After being selected as the OSP grantee, WSF struggled with developing 
and implementing accountability mechanisms and internal controls in the 
management of federal and foundation grants that increased from 
$150,000 in fiscal year 2004 to $12.9 million in fiscal year 2006. Further, 
with the addition of OSP to its operations, WSF was challenged to find 
systems and approaches for community outreach, school placement, 
family and school support, and financial management while implementing 
the program. Due to its need to quickly implement the program, WSF had 
little time to develop internal control policies and procedures, systems, 
and internal control activities needed to provide accountability over the 
use of grant funds for this new program.® WSF had to rely on the 
knowledge of its staff to perform the daily operations for OSP, and its 
abihty to rely on its key staff was greatly affected by high staff turnover. 
The discovery of fraud allegedly perpetrated by a WSF employee affecting 
one of its private scholarship programs highlighted the need for 
strengthened internal control activities. The lack of comprehensive 
policies and procedures, the use of nonintegrated systems, and the 
outsomcing of basic bookkeeping functions increased risk and weakened 
WSF’s accoimtability over OSP. Moreover, we found that WSF did not 
consistently adhere to the basic accountability and internal control 
procedmes it had established for scholarship payments. In reviewing OSP 
scholarship payments, we found that WSF’s practice of paying for tuition 
scholarships to schools that do not normally charge students tuition is a 
violation of the Act because the Act provides that the tuition charged OSP 
students cannot be more than the tuition charged for similarly situated 
non-OSP students. Furthermore, we could not determine whether 
payments for before-and-after care programs were appropriate because 
the documentation provided by WSF did not indicate that it had verified 
that the care programs met the criteria for determining whether a fee is 
allowable. Based on the documentation provided by WSF, we could not 
determine that WSF verifies that before-and-after school care programs 
are tied to the student’s academic program and part of customary fees 
charged by the school, which are Education and WSF’s key criteria under 
the Act for determining whether a fee is allowable. 



^Accountability can be defined 8is an organization’s ability to effectively demonstrate to 
both internal and external parties that resources are managed properly, programs are 
achieving intended goals and outcomes, and programs are being provided effectively and 
efficiently. To achieve accountability, organizations need internal control activities that 
provide reasonable assurance that (1) operations eire effective and efficient, (2) financial 
reporting is reliable, and (3) compliance with laws £md regulations is being achieved. 
GAO/AIMD-00-21.3.1. 
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WSF’s Rapid Growth and 
High Staff Turnover 
Created Accountability 
Risk for the Opportunity 
Scholarship Program 



Following its selection as a grantee in March 2004, WSF rapidly expanded 
its scholarship operations to provide OSP scholarships to students for the 
upcoming 2004-2005 school year/ This rapid expansion can be seen in the 
increase of WSF’s annual federal and foundation grants from $160,000 in 
fiscal year 2004 to $8.6 million in fiscal year 2005, and to $12.9 million in 
fiscal year 2006. (See table 3.) WSF, charged with running OSP, had to 
create systems for community outreach, school placement, and family and 
school support while implementing the program, and WSF faced the 
challenge of accomplishing school search and enrollment activities before 
the begirming of the 2004-2006 school year. WSF was also challenged to 
find ways in which thousands of OSP checks could be correctly generated, 
accounted for, and monitored for compliance with statutory requirements. 
With this rapid expansion, WSF had little time to design and implement the 
needed systems, policies, procedures, and internal controls for managing 
such a major increase in its operations. While WSF was able to process the 
OSP scholarship payments for school years 2004-2006 and 2006-2006, it 
used systems that were not integrated, resulting in data being manually 
transferred, and thereby increasing the risk of error. Also, WSF’s basic 
bookkeeping and accounting functions, including the printing of OSP 
scholarship checks and preparation of bank reconciliations, were not 
integrated, contributing to the fragmentation of WSF’s OSP operations and 
weakened accountability. 



^OSP is the largest of WSF’s scholarship programs. WSF currently provides scholarship 
funds to about 2,400 students, over 1,800 of which receive OSP scholarships. 
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Table 3: Growth in WSF from Fiscal Years 2003 — 2006 


Financial statement line Item 




Fiscal year ending June 30 




2003 


2004 


2005 


2006 


Federal and foundation grants* 


b 


$150,000 


$8,563,995 


$12,893,898 


Contributions and grants 


$1,801,251 


$1,929,707 


$1,576,706 


$1 ,793,409 


Total 


1,801,251 


2,079,707 


10,140,701 


14,687,307 


Scholarships expense” 


1,967,480 


2,268,942 


9,043,594 


14,108,731 


Expenditures of federal awards 


d 


d 


6,570,407 


11,664,898 



Source: WSF audited financial statements for fiscal years ending June 30, 2003, 2005, and 2006. 



“While OSP is a major federal program for WSF, WSF also has received a 2-year “Next Generation” 
grant from the Corporation for National and Community Service. This 2-year grant is designed to help 
organizations build programs in community service and volunteerism. 

“Beginning with fiscal year 2004, WSF reclassified contributions and grants for comparative purposes 
to show the federal and foundation grants separately from contributions. 

“This expense includes scholarships paid by WSF under both its publicly funded OSP and its 
Signature Scholarship Program, which is privately funded. 

“Single audits, which include the schedule of expenditures of federal awards, are required for grant 
recipients who expend $500,000 or more in federal awards in each year. These amounts are not 
available because WSF expended less than $500,000 in federal grants and therefore was not 
required to submit a single audit report for fiscal years 2003 and 2004. 

WSF’s rapid growth, the changing nature of its systems and processes for 
OSP scholarship payments, and the lack of comprehensive and detailed 
procedures meant that WSF had to rely on the knowledge of key staff to 
perform daily operations. However, the ability of WSF to rely on 
institutional knowledge was greatly affected by staff turnover in key 
positions. (See fig.l.) The Chief Financial and Operations Officer resigned 
in May 2006. WSF’s Accounting and Grants Manager resigned in January 
2007 and the Director, Finance and Operations resigned in February 2007. 
The Senior Director Education and Outreach left WSF during that time. 
The President and Chief Executive Officer then left in May 2007. The 
Senior Director for Scholarship Operations — a key person who 
understood WSF’s scholarship payment processes for the OSP — resigned 
in June 2007. Staff turnover at WSF, combined with the lack of established 
accountability procedures and systems, has significantly increased WSF’s 
overall fiscal and program operations risk. 
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Figure 1 : WSF Staff Turnover between May 2006 and June 2007 Based on Positions 
in Piace in August 2006 




Employee turnover between May 2006 and June 2007 



Source: Washington Scholarship Fund. 



WSF has recently increased its staff significantly. A comparison of figure 1 
and figure 2 shows this increase in staff resources from 11 staff in May 
2006 to 17 staff in August 2007. Staffing is supplemented by interns who 
work with the full-time staff on both of WSF’s scholarship programs. 



Page 16 



GAO-08-9 District of Coiumbia Opportunity Scholarship Program 









Figure 2: Reporting Structure Among WSF Staff as of August 2007 




Source: Washington Scholarship Fund. 



WSF’s Accountability Was 
Further Weakened by a 
Lack of Comprehensive 
Fiscal Policies and 
Procedures and Use of 
Nonintegrated Systems for 
Basic Bookkeeping and 
Accounting Functions 



WSF’s accountability over OSP scholarship payments was further 
weakened by a lack of comprehensive and detailed policies and 
procedmes to guide its processing of receipts and payment of scholarships 
and administrative expenses. For the fiscal year ended Jime 30, 2005, 
WSF’s external auditor reported that WSF was in the process of 
developing a formal accounting manual documenting its procedures. The 
auditor stated in a letter to WSF management that this type of accounting 
manual is necessary to ensure that transactions are treated in a 
standardized manner. In its letter, the auditor also noted that written 
procedures, instructions, and assignments of duties will prevent or reduce 
misunderstandings, errors, wasted effort, duplicated or omitted 
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procedures, as well as other situations that can result in inaccurate or 
untimely accounting records. The auditor recommended that WSF include 
in its accounting manual descriptions of each fiscal procedure — including 
payment of invoices, maintenance of accounts receivable subsidiary 
ledgers, and payroll procedures. 

In February 2007, WSF provided us with a copy of its Internal Systems 
Review, which was still in draft form and provided only general process 
descriptions. The document was created in response to the 
recommendation from its external auditor. WSF officials told us that its 
operations team would be continually writing and editing this document to 
further encourage organizational and financial integrity in its daily 
operations. The officials added that they expected the fiscal policies and 
procedures document to be final once WSF’s new nonprofit financial 
management software is operational. As of late June 2007, WSF officials 
stated that this new software would be operational for processing the 
2007-2008 scholarship payments. The February 2007 draft provides a high- 
level description of the processes for receipts, OSP and other scholarship 
payments, administrative expenses, human resources, and budgeting/ 
bank reconciliations. However, without extensive on-the-job training, the 
descriptions in this document are not adequately detailed for use in 
carrying out the procedures or for use in holding individuals responsible 
for key internal controls and accountability. 

The policies described in the draft Internal Systems Review are not 
specific and do not include controls over key activities for handling WSF’s 
funds and payments or the specific internal controls needed to help ensure 
that key control activities are being carried out. For example, the 
procedures state that bills submitted by WSF’s vendors are to be approved 
by the financial team, but the procedures do not include how the approval 
is to be documented and how the payment of the invoice is to be recorded 
in WSF’s accounting software. Also, in the section on procedures for 
sending the OSP scholarship payments data to the contractor for printing 
checks, there is no mention of controls and procedures to verify the 
accuracy of the payment amounts and the printed checks. While 
segregation of duties® is mentioned in the Internal Systems Review 



®The concept of segregation of duties can be defined as the division or segregation of key 
duties and responsibilities associated with financial operations among different members 
of the staff to reduce risk of error or fraud. For example, the responsibilities for authorizing 
transactions, processing and recording them, reviewing the transactions, and handling any 
related aspects should be performed by different persons. No one individual should control 
all key aspects of a transaction or event. 
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document for processing receipts and for issuing checks to WSF’s 
vendors, these procedures do not clearly outline responsibilities for (1) 
authorizing transactions, (2) processing and recording them, (3) reviewing 
the transactions, (4) handling any related access to cash, and (5) how 
these duties are to be separated. The control achieved through segregation 
of duties is important for small as well as large organizations. Where the 
staff is not large enough to segregate all duties, a compensating control, 
such as supervisory review, can be established. Segregation of duties is 
among the controls designed to prevent such fraudulent acts as those 
allegedly committed by a former WSF staff member in processing WSF’s 
private scholarship fimds (no OSP fimds were involved, according to WSF 
officials) between October 2006 and January 2007. The manager diverted 
funds from WSF’s Signature Scholarship Program (SSP) bank account and 
made personal charges to a WSF credit card. A WSF official subsequently 
identified the fraudulent activity through reviews of bank reconciliations 
and credit card charges. WSF management told us that it has implemented 
additional controls to help ensure that access to any accounts or funds is 
secure and that these controls will be tested as a part of WSF’s next 
financial statement audit. 

WSF processed OSP scholarship payments using systems that were not 
interfaced, resulting in data being manually transferred, a procedure that 
increases the risk of error. For school years 2004-2005 and 2005-2006, due 
to limitations in WSF’s accounting system, WSF used an Excel spreadsheet 
to track information about individual OSP scholarship payments. 

However, the detailed payment and check information in the Excel 
spreadsheet could not be uploaded or interfaced with the accounting 
software used by WSF. As a result, only the lump sum totals for batches of 
payments were recorded in the WSF accounting software, not the 
individual scholarship check information. This system was used as the 
basis for WSF’s financial reports. 

WSF enhanced its scholarship payment process for 2005-2006 through the 
implementation of a Web-based, online billing system that enabled 
participating schools to electronically enter the tuition and fee charges 
and submit invoices for all OSP students attending that school.’’ WSF also 
created a student database that included key information about each 



^Schools entered the tuition and fee charges for each student into WSF’s online billing 
system at the beginning of the school year. Some of the fees commonly included were for 
uniforms, field trips, before-and-after care, cafeteria services, and transportation. 
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student. However, because the online billing system was not integrated 
with the student database, key information about each OSP student had to 
be uploaded electronically into the online billing system to enable 
participating schools to submit invoices. 

WSF’s overall process for key bookkeeping and accounting functions, 
such as check printing and bank reconciliation, was not well defined or 
integrated. Further, these processes were outsourced, requiring 
monitoring and oversight of contractor’s work to ensure against errors. 
For example, the bank reconciliation process — a key cash management 
control — ^was made more difficult by the tracking of individual payments 
in the Excel spreadsheet or student database, neither of which was 
integrated with the accounting system. The lump sum recording of 
payments also made it difficult to track the separate manual entries in its 
accounting software for voided checks, reissued checks, and other 
adjustments, because individual OSP scholarship checks were not directly 
matched with corresponding entries. Therefore, the preparation of bank 
reconciliations was difficult. In 2007, WSF officials noted errors in recent 
bank reconciliations. These errors, combined with WSF’s detection of 
fraud allegedly perpetrated by the former WSF employee in its nonfederal 
program, have prompted WSF to begin a process of redoing all bank 
reconciliations, starting with 2004. As of the end of our field work, this 
effort had not been completed and therefore the outcome is unknown. 

WSF officials have contracted to purchase a financial management 
software package designed for nonprofit organizations that they believe 
will facilitate the recording of the individual scholarship checks. 
According to WSF, the system should be in place for the 2007-2008 school 
year. As WSF continues to implement integrated systems and processes, 
the need for manual data transfers will be reduced. 



WSF Did Not Consistently 
Adhere to Basic 
Accountability and 
Internal Control 
Procedures 



WSF did not consistently adhere to its established OSP scholarship 
payment procedures and made payments without the required 
documentation or approvals, and it did not fulfill its oversight role for 
ensuring that participating schools are financially responsible. Lack of 
appropriate accountability and internal control increases the risk that 
program funds will not be used in accordance with program requirements. 



On the basis of our examination of supporting documentation for 
payments for a random sample of 50 students receiving scholarships at 
25 schools in school year 2005-2006, we found the following problems with 
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scholarship payments made for 46 of the 60 students. Payments for some 
students had more than one of these problems. 



• For 39 of the 50 student files we reviewed, WSF had paid fees based on 
incomplete fee approval forms from the participating schools, which 
lacked authorizing signatures from the participating school, dates, or 
both. Participating schools submit these forms to obtain WSF’s 
approval of fees that are not included in the school’s published list of 
tuition and fees. According to WSF’s procedures, these forms must be 
reviewed and approved by WSF officials before payment can be made. 
Without having properly approved fee information, WSF is at risk of 
paying unapproved fees. 

• For 9 of the 50 students, we found related student or school files were 
incomplete. Some of the items that were missing included 
disbursement and payment detail reports as well as other documents 
such as school placement letters. Disbursement reports provide a 
summary of the tuition and fees that have been billed to the student’s 
scholarship account for the year. Payment detail reports list all tuition 
and fees for each OSP student attending the participating school, and 
the correctness of the information on these reports is to be certified by 
officials of the participating schools. Without this information, we 
could not assess whether the controls were operating as intended. 

• In 23 of the 60 student files we reviewed, disbursement reports did not 
include the WSF student identification number next to the student’s 
name or the student identification or name was incorrect. Although 
WSF maintains that its student identification numbers are not used as 
key identifiers, these student numbers are the only numbers that link 
the disbursement reports and the scholarship checks. Without a 
consistent student identifier to link students in the program with the 
actual payments being made, WSF is at risk of losing accountability for 
specific payments made for individual students in the program. 

• We found a scholarship payment that WSF paid for a fee that was not 
included on the payment detail report and a fee payment for another 
student that was based on a payment detail report that had not been 
certified by a school official. Without this certification, WSF officials 
did not have assurance from the participating school that the payment 
was made based on correct and complete information. 
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Under the Act, the grantee is to use the funds to provide eligible students 
with scholarships to pay tuition, fees, and transportation to enable 
students to attend the D.C. private school of their choice. Neither the Act 
nor its legislative history defines the terms “tuition,” “transportation,” or 
“fees.” According to Education officials, the Department left the 
determination of what fees are to be included in a scholarship payment to 
WSF. WSF established the policy that any fee for an activity, service, or 
product that contributes to the academic success of a student is allowable 
if two key criteria are met. One criterion is that fees paid must be for 
services offered to all similarly situated students, not just OSP students. 
The other criterion was that the rates of all fees for OSP students must be 
the same as rates charged for similarly situated, non-OSP students. WSF 
commonly uses OSP scholarship funds to pay fees for school uniforms, 
books, cafeteria services, tutoring, before-and-after care, physical 
education, and enrichment (music, dance, and art). 

During the 2006-2007 school year, WSF offered scholarships up to $7,500 
to students enrolled in schools that do not normally charge students 
tuition. For school year 2006-2007, WSF offered such scholarships to about 
30 students. At these schools, families of non-OSP students typically pay a 
small monthly fee as a sign of commitment to the school, but tuition and 
other expenses are supported by private donations to the school. OSP is 
providing scholarship payments of $7,600 per year to pay tuition for OSP 
scholarship children to attend these schools while the families of non-OSP 
students pay no tuition. Section 307(a)(1) of the Act requires that the 
grantee ensure that the amount of any tuition or fees that the school 
customarily charges OSP students not exceed the amount of tuition or fees 
that the school customarily charges non-OSP students. In reviewing OSP 
scholarship payments, we found that WSF’s practice of paying for tuition 
scholarships to schools that do not normally charge students tuition is a 
violation of the Act because the Act provides that the tuition charged OSP 
students cannot be more than the tuition charged for similarly situated 
non-OSP students. The small monthly fee, because it is charged to all 
students, would be an allowable fee under the OSP. WSF maintains that 
there is no violation of the Act because students attending these schools 
tuition-free are supported by scholarship dollars in the same way as those 
who receive OSP funds. 

During school year 2005-2006, WSF was billed for before-and-after school 
care for 31 of the 60 students whose payments we reviewed. On the basis 
of the documentation that WSF provided to us for the 26 schools in our 
sample that offered before-and-after care, in 22 cases we could not 
determine whether these services included any academic support 



Some Payments Are Not in 
Compliance with the Act, 
and Others Raise 
Questions about 
Compliance 
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activities. Absent this characteristic, before-and-after care would not meet 
the requirement that a fee must contribute to a student’s academic 
success. 



While OSP Students 
Generally Met the 
Program’s Eligibility 
Requirements, WSF 
Had Limited Success 
Meeting Recruiting 
Priorities and Did Not 
Provide Parents 
Complete Information 



WSF had limited success in recruiting students from schools in need of 
improvement, ensuring private school placement opportimities and 
quality, and providing parents accurate information on private schools. 
Students who were offered OSP scholarships generally met the program’s 
income eligibility requirements, and applicants from schools in need of 
improvement had priority for receiving scholarships, but low numbers of 
scholarship recipients came from such schools. ’While a large number of 
schools were recruited to participate, they varied widely in the numbers of 
placements they made available and characteristics associated with 
quality, such as teachers’ qualifications, and information WSF provided 
parents was not always complete and accurate. 



Grantee Recruited Eligible 
Students, but Low 
Numbers Were from 
Schools in Need of 
Improvement 



Data reported to the grantee generally indicated students who were 
offered scholarships met the financial eligibility criteria. Despite 
considerable outreach efforts and priority given to students from schools 
in need of improvement, they were underrepresented among recipients of 
scholarship offers relative to their presence in the population of District 
students, owing to both challenges in recruiting applicants and school 
openings. 



Under the terms of the memorandum of understanding between Education 
and the District of Columbia regarding the operation of the program, the 
District, in consultation with Education, agreed to identify a community- 
based nonprofit organization to conduct activities, including outreach to 
parents and private schools, prior to the award of the grant in order to 
promote a successful launch of the program for the 2004-2005 school year. 
This entity — Fight for Children — met with private school officials, 
collected preliminary information about these schools, and initiated 
community outreach activities. After the grant was awarded, WSF 
assumed responsibility for these efforts and built upon them by engaging 
in such activities as holding community meetings, running radio 
advertisements, conducting mail campaigns, and reaching out to minority 
communities. However, WSF encountered obstacles in recruiting students 
from schools in need of improvement, and less than a quarter of all 
students who were offered scholarships by 2006 came from these schools. 
In fact, 4 percent of students who were offered scholarships the first year 
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were from schools in need of improvement, while about 11 percent of 
District students attended regular public and charter schools in need of 
improvement that year. Table 4 describes some of WSF’s outreach 
activities, and figure 3 shows examples of some outreach materials used. 



Table 4: Types of Outreach Methods Used to Reach Potentially Eligible Families 



Method 


Examples 


Paid advertising 


• Direct mail 

• Newspapers 

• Public transportation ads 

• Movie theater advertisements 

• Radio 


Neighborhood meetings at a variety of 
locations 


• District of Columbia Armory and convention center 

• Community centers 

• Churches 

• Housing developments 

• Libraries 

• Public housing 

• School-sponsored events 

• Homes 


Direct mail to targeted audiences 


• Temporary Assistance for Needy Families recipients 




• Families with students attending District of Columbia public schools designated as in 
need of improvement 




• Commercial list of 27,000 low-income families 

• District of Columbia Public School and charter school mailings 


Posters at a variety of locations 


• Community centers 

• Clinics 

• Churches 

• Homeless shelters 

• Government services intake offices 

• Outreach centers targeting minority populations 

• Retail outlets (grocery stores, beauty salons) 


Flier distribution at a variety of locations 


• Schools in need of improvement 

• Public transportation stops 

• Targeted neighborhoods 

• Retail outlets (grocery stores, beauty salons) 


Office accessibility 


• Evening and weekend hours 

• Satellite offices 

• Bilingual interpreters, primarily Spanish 

• Volunteers 

• School and application fairs 



Source: GAO summary of WSF information. 
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Figure 3: Examples of Outreach Materials Grantee Used To Recruit Low-Income Families 



BECAS PARA ASISTIR A UNA ESCUELA PRIVAOA EN EL 
DISTRITO DE COLUMBIA EN EL ANO ESCOLAR 2005-06 

TUITION TO ATTEND O.C. PRIVATE SCHOOLS IN 200S-200A 
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Apply on February 22 , 4-7 p.m. 

Thurgood Marshall Center 

1816 12"’ Street, NW 

Opportunity Scholarships 

of up to $7,500 

to attend a D.C. private school In 2007-2008 
for public school children going into kindergarten 
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Various factors hindered WSF’s efforts to recruit more students from 
schools in need of improvement for the first year, according to WSF 
officials. Officials said that many families were skeptical about the 
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program — stating that it was “too good to be true” or that private schools 
would be reluctant to accept their children — and had heard 
misinformation, such as that receiving a scholarship could reduce social 
service benefits. In addition, WSF officials said they only had a short time 
after receiving the grant to recruit a large number of potentially eligible 
families and help them through the complicated application and school 
admissions process before they had to conduct the first lottery. 

By the second school year, 2005-2006, the percentage of students from 
schools in need of improvement who were offered scholarships increased 
to 38 percent. According to WSF officials, several factors contributed to 
this growth; More families knew about the program and were less 
skeptical than they had been in the beginning, and WSF had more time to 
recruit eligible families. In addition, the percentage of children who 
attended District public and charter schools designated as in need of 
improvement increased, therefore enlarging the pool of potentially eligible 
applicants from these schools. 

WSF decided to reduce the extent of its outreach efforts for the third year, 
2006-2007, because of the limited availability of new scholarships for 
students. “ Most of the grant funds were needed to support students 
awarded scholarships in years one and two. According to WSF officials, 
conducting extensive outreach efforts when only a limited number of 
scholarships were available would have unrealistically raised some 
families’ hopes of receiving scholarships. WSF cut back on media 
advertising, but mailed flyers to families with children in schools in need 
of improvement and held community meetings and events, such as school 
fairs. “ About 24 percent of students who were offered new scholarships 
for the third year were from schools in need of improvement. (See fig. 4.) 

Lotteries for scholarship awards incorporated a mechanism to afford 
required priority to students from schools in need of improvement, but an 
applicant’s probability of selection was also influenced by the number of 
private school openings offered at his or her grade level. This is significant 



^°In addition, WSF only allowed students entering grades kindergarten through grade 5 to 
apply for scholarships for the 2006-2007 school year because of the shortage of high school 
openings. 

^'Each year, WSF has included a brochure on OSP with other information on school choice 
that the District of Columbia Public Schools or State Education Office has mailed to 
families with children in District public schools. 
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because private school openings were most plentiful at the elementary 
level, while improvement designations had been more common among 
middle and secondary schools. In order to give priority to students 
attending schools in need of improvement, the evaluator assigned highest 
probabilities to those students that attended District schools with this 
designation, lower priority to students from public schools without this 
designation, and, for the first year of operation, lowest priority to students 
from private schools. Because the lottery also awarded scholarship offers 
in proportion to the number of possible openings reported by participating 
schools at particular grade levels, students in kindergarten through grade 8 
had considerably better chances of receiving scholarship offers than high 
school students because many more openings were available for them.^^ 
While District middle and secondary schools were more likely to be in 
improvement status than elementary schools, private school openings at 
the secondary level were so scarce that WSF restricted application in later 
years of the program to students in the elementary grades. 



^^During the third year, WSF only allowed students entering grades kindergarten through 
5 to apply because there were few placements available at the high school level and 
students in cohorts 1 and 2 were matriculating into these grades. During the fourth year, 
WSF limited applications to students in grades kindergarten through grade 7. 
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Figure 4: Percentages of Students Offered OSP Scholarships Who Attended 
Schools Designated as in Need of Improvement Were Consistently Below the 
Percentages of Students in District of Columbia Schools Who Attended Such 
Schools 

Percent 

100 

90 

80 

70 

60 



51 52 




School/program year 



Percentage of students offered OSP scholarships who were from schools in need of improvement 

Percentage of District public and charter school students enrolled in schools in need of improvement 

Source: GAO analysis of data from WSF and the District of Columbia Public Schools. 

Note: The District of Coiumbia issued its iist of schoois in need of improvement each August for the 
prior school year. Because WSF and the evaluation contractor had to conduct scholarship lotteries in 
the spring before the District issued the lists, they had to use prior year designations. For example, 
they conducted the scholarship lotteries for the 2004-2005 school year in spring 2004 but had to use 
the list of schools designated as in need of improvement in 2002-2003 because the District did not 
issue the 2003-2004 school list before the end of the school year. Accordingly, in the figure above, 
both bars in each year represent students from schools that were designated as in need of 
improvement in the previous August. 



Of the 2,845 students who were offered scholarships during the first 
3 years of the program, most were African-American, but some were 
Hispanic, Asian, and white students, and almost all met the income 



cannot report specific demographic characteristics of students offered scholarships 
because the data we received from both WSF and Westat contained large amounts of 
missing data. 
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eligibility requirements. “ About 77 percent had attended District public 
schools, with the remainder split among students who had attended 
private schools, students who had not previously attended school, and 
students from public and private schools outside the District. Most 
students — about 80 percent — were enrolled in elementary and middle 
schools,'^ while about 17 percent were enrolled in grades 9-12. 

Although most students used their scholarships when they received them, 
many did not or did not continue to use them after the first year, 
potentially freeing up scholarship money for some additional scholarship 
offers, but students who had been offered scholarships could decide to 
use the scholarships at any time during the program’s operation by 
enrolling or re-enrolling in a participating private school. About 75 percent 
of all students who received scholarships during the first 2 years used 
them at some point during the time they were available to them. Among 
students offered scholarships in the first year the program operated 
(cohort 1), 68 percent initially used their scholarship, and of those who 
used them initially, 69 percent used their scholarships all 3 years that the 
scholarships were available to them and for which data had been 
collected. Among students who were first offered scholarships in the 
program’s second year of operation (cohort 2), about 71 percent initially 
used their scholarship, and of those who used them initially, 78 percent 
continued to use their scholarships for the next 2 school years. One 
hundred eleven students in cohort 1 and 5 students in cohort 2 did not use 
their scholarships when first available, but did so eventually. 

WSF had little data that described reasons students did not use their 
scholarships when they first received them or continue to use them for 
subsequent school years, but anecdotal information was available. The 
reasons reported by some parents related to family issues, such as 
personal problems, moving, and special education needs, while others 



^''During the first year of the program, WSF reported that an audit had found a small 
number of students erroneously received scholarship offers whose household incomes 
exceeded 185 percent of the poverty level at the time of application and who were, 
therefore, not financially eligible for the program. WSF indicated it subsequently used 
funds from its private scholarship program to allow these students to continue private 
school participation. 

^^With the approval of Education’s General Counsel, WSF awarded some scholarships to 
children whose parents planned to place them in a participating preschool or 
prekindergarten as long as the children satisfied the District’s age requirements to enter 
kindergarten and the preschool was best fitted to meet their needs. 
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were more logistical, such as lack of transportation, before- or after-care, 
or a convenient school. Parents also chose to enroll their children in 
District charter schools, neighborhood schools, or other private schools 
that did not participate in the program. According to the Parent and 
Student Voices May 2007 report,'® some parents also reported several 
factors that hindered their children’s ability to use the scholarship, such as 
losing their scholarship because the family’s income exceeded the 
previous 200-percent poverty threshold for renewing scholarships,'^ a 
dearth of openings at the high school level, and receipt of a scholarship for 
only one child in a sibling group. 

Scholarship usage rates also varied according to the type of school the 
student previously attended and students’ needs for special educational 
services. The program evaluation found that fewer students from schools 
in need of improvement and from high schools in general used their 
scholarships than other students. In addition, fewer students that reported 
having a learning or physical disability used scholarships than other 
students. 



'^Stephen Q. Cornman, Thomas Stewart, and Patrick Wolf, The Evolution of School Choice 
Consumers: Parent and Student Voices on the Second Year of the D. C. Opportunity 
Scholarship Program (Georgetown Public Policy Institute, May 2007). 

"^Under the rules of the program, the household income of applicants could not exceed 
185 percent of the poverty level at the time of application, but participants could continue 
in the program as long as their household income did not exceed 200 percent. In December 
2006, Congress passed legislation that raised the limit on household income for students 
who had enrolled during the program’s first 2 years from 200 percent of the federal poverty 
level to 300 percent of the federal poverty level. This allowed K-12 students who would 
have “earned out” of the program to maintain their eligibility. 
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Participating Schools 
Ranged Widely in the 
Number of Openings 
Available to Scholarship 
Recipients, and Few 
Openings Were Available at 
the Secondary Level 

Some students who received scholarships had limited choices, particularly 
students in the upper grades and those who wanted to attend a secular 
school. For example, according to WSF data for school year 2005-2006, 
only about 70 openings were available at the high school level (compared 
to about 650 for students in kindergarten through grade 5 and about 
200 for students in grades 6-8). The majority of scholarship students 
attending high schools went to one religious school, and WSF raised 
private funding to pay for the tuition above the scholarship cap. In 
addition, students who desired a secular school had a limited number to 
choose from, since most of the participating private schools were Catholic 
or Protestant, and these schools offered most of the openings. The 
remaining schools included some that were Afro-centric or Muslim, or 
offered only early childhood education. 

About 88 percent of all scholarship users attended schools with tuitions 
below the $7,500 cap. Although tuition rates varied, only 3 percent 
attended the most expensive schools that charged $20,000 or more, 
perhaps because these schools or private donations could not make up the 
difference between the tuition and the $7,500 cap or because these schools 
had financial aid programs of their own on which they could draw that 
were not subject to OSP financial and residential eligibility requirements. 
In general, schools offering high school placements had higher tuitions 
than other schools. (See table 5.) 



The 70 District of Columbia private schools that have participated in the 
program varied in terms of their characteristics and the number of OSP 
students they enrolled.^® Students who received scholarships ranged from 
none to 67 percent of the enrollment of participating private schools. For 
example, in 2006-2007, more than half the scholarship students were 
concentrated in 19 schools. Sixteen schools had accepted between 0 and 
5 scholarship students. Four schools accepted only students who they had 
previously accepted. 



^According to the National Center for Education Statistics, the District of Columbia had 
87 private schools as of the 2003-2004 school year; 70 of these schools have participated in 
OSP, but not all schools have participated in all years. The schools that are not 
participating include some that are highly specialized, such as a ballet school, and some 
that serve only students with severe disabilities. 
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Table 5: Summary of Tuition Rates by Grade Level for Participating Private Schools, School Year 2006-07 



Elementary 


Elementary and middle 


Elementary through high school 


Number Number of 


Number Number of 


Number Number of 



Tuition 


of schools 


OSP students 


of schools 


OSP students 


of schools 


OSP students 


$ 2 , 100 - 5,000 


4 


76 


18 


999 


1 


67 


$ 5 , 001 - 7,500 


4 


95 


7 


190 


2 


97 


$ 7 , 501 - 10,000 






2 


15 






$ 10 , 001 - 15,000 






1 


5 






$ 15 , 001 - 20,000 


2 


4 


1 


25 






$ 20,000 - 1 - 


1 


1 


1 


3 


3 


38 


Total 


11 


176 


30 


1,237 


6 


202 
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Middle or junior high 




Middie and high schooi or 
high schooi oniy 




Aii schoois 




Number 
of schoois 


Number of 
OSP students 


Number 
of schoois 


Number of 
OSP students 


Number 
of schoois 


Number of 
OSP students 














23 


1,142 














13 


382 


1 




15 


1 




72 


4 


102 


1 




12 


2 




14 


4 


31 


1 




3 


4 




5 


8 


37 








1 




2 


6 


44 


3 




30 


8 




93 


48 


1,738 


Source: GAO analysis of WSF data. 



Note: In addition, three students attended preschools with tuition between $7,501 and $10,000, and 
two students attended a preschool with tuition between $5,001 and $7,500. 

While the Act requires WSF, as the grantee, to ensure that participating 
schools are financially responsible and are using the received funds 
effectively, WSF did not fulfill its oversight role of ensuring that 
participating schools are financially stable. WSF designed letters of 
agreement that detailed the requirements for participating schools, 
including the need to show evidence of financial stability on an annual 
basis. However, for all 25 schools attended by the 50 students in our 
random sample, we did not find certain documents that should have been 
submitted by the schools showing financial stability or that a financial 
audit had been completed. 

Proof of financial stability is critical for providing assurance that a 
participating school can maintain operations as an ongoing entity and that 
OSP students can continue enrollment in those schools. During school 
year 2006-2007, one school that enrolled 25 scholarship users announced 
closure due to bankruptcy. Two schools that were members of the Center 
City Consortium, a group of Catholic schools run by the Catholic 
Archdiocese of Washington, closed in school year 2007-2008 due to 
financial problems, but the 134 scholarship users affected by the closures 
were offered placements in other consortium schools. Presently, the 
archdiocese is exploring the possibility of converting 8 of the 12 schools in 
the consortium to charter schools by school year 2008-2009. 
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The extent to which private schools reported characteristics associated 
with high-quality educational programs varied. We found that, as reported 
in the School Directory, D.C. Scholarship Program: 2004-05,^^ at least 3 of 
52 schools that participated that year indicated that at least half of their 
teachers did not have at least a bachelor’s degree, and 6 schools indicated 
that about 10 to 20 percent of their teachers lacked at least a bachelor’s 
degree. Further, many of the schools were not accredited, and there is no 
evidence that they submitted evidence of educational soundness 
acceptable to the Board. 

The MOU and the Act limit participation to private schools operating 
lawfully in the District. Yet, neither Education, the District of Columbia, 
nor WSF ensure that all participating private schools meet basic 
requirements for operating legally in the District of Columbia, and some 
schools have not met these requirements. WSF relies on school officials to 
self-certify that they are operating lawfully, and the District had not 
reviewed any schools to determine whether they had met the District’s 
requirements. We selected a non-probability sample of 18 schools using 
various criteria, including whether or not schools had registered with the 
District’s Department of Consumer and Regulatory Affairs and whether or 
not schools were accredited by an agency recognized by the District and 
asked the District to supply copies of their occupancy certificates to us. 
District officials provided documentation indicating that 3 of 18 schools 
we selected for review for certificates of occupancy lacked them; 6 had 
permits that did not specify use as a private school, child development 
center, or before and after school care center; and 7 of the 18 appeared to 
have occupancy permits that designated use as child development centers 
with before and after school care for school-age children, but did not 
reflect the operation of a private school. One school where OSP students 
constituted over 60 percent of enrollment applied for an occupancy permit 
for operating as a private school in March 2007 after it had participated in 
OSP for 3 years. Because District officials could not find any reports on 
file, they concluded several schools may not have submitted required 
annual reports of operation providing basic information on their 
curriculum, teachers’ education, accreditation, and school facilities. These 
officials also reported that the District government had not previously 
played an active role in overseeing the operations of WSF. Figure 5 shows 



used the School Directory D.C. K-12 Scholarship Program: 2004-2005 School Year, 
(Washington, D.C.: Washington Scholarship Fund), for our analysis because the directories 
for the 2006-2007 and 2007-2008 school years appear to inaccurately report data on 
teachers’ degrees for many schools. 
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examples of schools participating in OSP that District officials reported 
did not have certificates of occupancy specifying use as an educational 
facility on file with the Department of Consumer and Regulatory Affairs. 



Figure 5: Examples of Schools That Operated without a Certificate of Occupancy 
Specifying Use as a Private Day School 




Source: District of Columbia government. 



According to WSF officials, they had conducted school visits at 42 of the 
58 schools with OSP students. However, we found only one written report 
documenting a visit to one of these schools. Selection of schools to visit 
was based partly on certain triggers, such as parent complaints. WSF 
officials told us that they had several versions of a school review form 
used to document visits and provided us a copy of the form used at the 
beginning of the 2005-2006 school year. However, they informed us that 
the use of this form had been discontinued shortly afterward. Site visits 
such as WSF’s school visits are an important control, especially given that 
schools self-certify that they meet all regulations established for private 
schools imder District law. Self-certification without review to verify that 
the certifications are factual increases the risk that federal funds intended 
to allow children from low-income famihes to attend private schools will 
result in some students attending schools that are not in comphance with 
District law. 
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Information WSF Provided 
to Help Parents Select 
Private Schools Was 
Incomplete and Contained 
Inaccuracies 



The Act required the grantee to provide parents with irrformation that 
would help them make informed decisions about selecting a school for 
their child. The MOU between Education and the District of Columbia 
provided that the grantee was to provide certain specific information 
about the school to parents, including the qualifications of the school’s 
teachers, the achievement of its students, and its safety and environment. 
The provision of information is important since many students who were 
offered OSP scholarships had little experience with private schools. 
Parents also said they would like to have information that would help 
them evaluate private schools.^" 



Although WSF compiled an annual directory to help parents during the 
selection process, it did not collect or omitted or incorrectly reported 
some information that would have helped parents evaluate the quality of 
participating schools.^^ WSF did not provide information about the 
achievement of each school’s students in order to help parents select 
appropriate schools for their children. However, it did require schools to 
certify that they had given parents of enrolled OSP students information 
on their children’s academic progress and overall school safety. 

WSF also requested some irrformation from the schools that it did not 
provide that could have helped parents during their decision-making 
process, most notably irrformation on a school’s accreditation status. In 
addition, some information WSF did provide to parents may have been 
misleading. Some schools reported to WSF that they had such facilities as 
a library or gymnasium, although the irrformation they reported to the 
District indicated no such facilities. 



Finally, WSF incorrectly reported information on some schools that could 
have significantly affected parents’ choice of schools, primarily the 
percentage of teachers who had at least a bachelor’s degree and tuition 



^°Stephen Q. Cornman, Thomas Stewart and Patrick Wolf, The Evolution of School Choice 
Consumers: Parent and Student Voices on the Second Year of the D. C. Opportunity 
Scholarship Program (Georgetown Public Policy Institute: May 2007). 

^'Each annual directory described the general characteristics of participating private 
schools, such as their enrollment, enrichment programs, and access to public 
transportation. 

^^The Act required that schools provide parents with information on enrolled students’ 
academic achievement, as measured by a comparison with the aggregate academic 
achievement of other students at the student’s school in the same grade or level on an 
annual basis. 



Page 36 



GAO-08-9 District of Columbia Opportunity Scholarship Program 




rates. For example, the 2007-2008 school directory reported that less than 
half of teachers in five schools and between 51 and 75 percent of teachers 
in another five schools had at least a bachelor’s degree, although the 
2004-2005 directory reported that 100 percent of teachers in these schools 
had at least a bachelor’ degree. In addition, the school directory for 
2007-2008 reported that the tuition for at least three private schools was 
between $10,000 and $15,000. However, in verifying tuitions listed in the 
directory by comparing them to tuitions cited in other sources, we learned 
these schools do not charge non-OSP students tuition and typically charge 
only a modest fee, such as $25 per month. The tuitions listed for these 
schools may have motivated parents to obtain scholarships in the belief 
that they could not otherwise afford them. 



Evaluation Contractor 
Adopted a Strong 
Methodology, but 
Program 

Implementation and 
Other Factors Limited 
the Usefulness and 
Generalizability of 
Findings 



The evaluation contractor adopted an evaluation design that used random 
assignment to determine which applicants were offered scholarships and 
which were not, but program implementation decisions and other factors 
limited the usefulness and generalizability of the evaluation’s findings 
regarding the effects of scholarship offers on the achievement of low- 
income, District public school students. The contractor adopted a random 
assignment evaluation design that enabled comparisons between students 
from District public schools who were offered scholarships (the treatment 
group) and those that were not (the control group) and was designed to 
measure the effect of scholarship offers on applicants who had attended 
public schools. The contractor used statistical techniques to estimate the 
effects of using such a scholarship. The methodology adopted was a 
strong experimental methodology that could have produced an evaluation 
that reflected statutory requirements. 



Many factors have limited the evaluator’s ability to make the achievement 
comparisons described in the statute and have complicated interpretation 
of the evaluation’s findings or their usefulness and generalizability. For 
example, changes in the District’s achievement testing program and the 



^^These schools were specifically designed to help low-income children and typically enlist 
financial contributions from institutional and individual donors in order to operate. Other 
than a modest fee, students are not required to pay any tuition or other costs. These 
schools may provide academic support, tutoring, extended school days, Saturday classes or 
enrichment activities, summer programs, uniforms, and free meals. 

^^As noted earlier in the report, we found that WSF’s use of OSP funds to pay tuition for 
students attending schools that do not normally charge students tuition is not in 
accordance with the Act. 
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scarcity of placements offered by participating high schools limited the 
evaluators’ ability to make comparisons that were of interest to Congress 
between students offered scholarships and their peers in District schools, 
including comparisons with regard to graduation rates, dropout rates, and 
college placement. In addition, although the Act directed the evaluator to 
determine the effects of the program on District regular public schools, the 
opportunities for choice within the public school sector grew over the 
same time frame, making it very difficult, if not impossible, to disentangle 
the effects of this program on regular public schools from the effects of 
charter schools, special schools, and magnet and special programs that 
serve far more students. Along with changes in the District’s testing 
program, limited number of scholarship placements at the high school 
level, and simultaneous growth of other choice options, other factors may 
have limited the evaluator’s ability to make requested comparisons and to 
detect statistically significantly differences and generalizable results. 

These factors include the inability to include some scholarship recipients 
in the evaluation, crossover between treatment and control groups, 
concentration of scholarship students in a subset of schools, and the 
combination of academic and social supports with scholarship offers. 
Given these differences, some comparisons relied on limited data. 

Changes in the District’s testing program. Changes in the District’s 
testing program made it impractical for the contractor to perform some 
comparisons that Congress had requested, greatly increased the cost of 
testing, and will likely continue to decrease the quantity and sufficiency of 
the follow-up achievement test data. At Education’s urging, the District 
changed its standardized achievement testing program in spring 2006 from 
a norm-referenced test to a criterion-referenced test to better comply with 
the requirements of the No Child Left Behind Act, but this had major 
effects on the evaluation. The norm-referenced test, like all norm- 
referenced tests, had compared students’ knowledge with respect to the 
knowledge of other students and sorted and ranked students according to 
how much they know in comparison to other students. Because students 
are ranked against each other, and normalized on a bell curve, norm- 
referenced tests are constructed so that scores at d iff erent grade levels are 
vertically equivalent and scores of children at various grade levels can be 
combined to draw comparisons. For example, a second and an eleventh 
grader who received a percentile score of 75 would have equivalent scores 
because both surpassed the scores of 75 percent of their grade peers. In 
contrast, the new criterion-referenced tests, like all tests of this type, 
measure whether students have mastered certain skills and concepts 
and/or what they have learned in certain grades or classes. Scores on 
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criterion referenced tests do not yield equivalent scores that can be readily 
combined to draw comparisons across grade levels. 

Although the Act directed the evaluator to compare the achievement of 
students offered scholarships with students of the same grade in public 
schools and the achievement of scholarship users with students of the 
same grade in public schools, the evaluation contractor could not 
reasonably or cost-effectively make these comparisons. The District’s new 
testing program meant a common test that yielded comparable scores for 
all grades, from the year eligible applicants entered the program through 
the duration of the program, was no longer available, and administering 
two testing programs to all students in the District would not be 
economically or educationally feasible. 

In addition, the change in the District’s testing program made it necessary 
for the evaluation contractor to expand the number of students it planned 
on testing from only those in the treatment group to also include those in 
the control group in order to make comparisons between these groups 
because students in the control group would no longer be participating in 
the norm-referenced testing as part of their public school experience. 
According to Education officials, this increased the cost of testing and 
outreach efforts associated with the testing to a total of $800,000 to 
$1 million per year. 

The change in tests will also make it difficult to make certain other 
potentially useful achievement comparisons, particularly with respect to 
students who do not use their scholarships and students in the control 
group. Although threats to withhold scholarship funds have been generally 
effective in securing the participation of scholarship users in testing 
sessions, no punitive measures are linked to a failure to participate for the 
other groups. The rewards offered — cash gifts, free tax preparation, and 
an opportunity to enter a lottery to receive a second chance for a 
scholarship — have not been sufficient to motivate many nonusers to 
attend Saturday testing sessions or to engage in appropriate test-taking 
behavior such as completing subtests and remaining for the duration of 
the test. Missing test data, as well as the number of students who fail to 
participate in testing, will likely increase over time, reducing the 
comparability of the treatment and control group. Although statistical 
techniques, such as weighting and imputation, can be used to adjust for 
missing scores, these techniques cannot guarantee findings would not be 
biased when the unweighted response rate — that is, the number of 
students for whom scores are available — ^falls to very low levels. (See 
table 6.) 
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Table 6: Unweighted Percentage of Students with Missing Achievement Test Scores at Baseiine and after 1 Year of Program 
Participation 



Rates in percent 

Reading Math Both reading and math 

Treatment Controi Treatment Controi Treatment Controi 

Missing achievement scores at baseiine 

Cohort 1 UA 23^3 1^0 23^8 iaO 2^ 

Cohort 2 2M sT!o 13^6 2Z7 1Z8 2^ 

Missing achievement scores after 1 year of program participation 

Cohort 1 21^7 4Z0 21^7 4Z0 21^7 4^ 

Cohort 2 23^6 3s!2 208 304 203 2^ 



Source: GAO analysis of Westat data. 



Scarcity of high school placements. The high schools participating in the 
program offered very few placements, thus limiting the number of high 
school students and graduates in the evaluation and hampering the ability 
of the evaluator to make statistically meaningful comparisons of dropout, 
graduation, and college admission rates of scholarship recipients and 
nonrecipients. (See table 7.) For example, according to data provided by 
WSF, the program graduated only 7 students from high school in its first 2 
years of operation, too small a number to make sound comparisons. In 
addition, the low numbers of secondary openings could indirectly 
exacerbate attrition from the evaluation if, as scholarship users advance 
through the grade levels, they find placements in higher grades are not 
available and become less motivated to participate in data collection 
necessary for the evaluation. 



Table 7: Number of Total Private School Placements for OSP Students 



Year 


Kindergarten 


Grades 1-5 


Grades 6-8 


Grades 9-12 


Total 


2004-2005 


296 


977 


397 


68 


1,738 


2005-2006 


169 


474 


192 


72 


907 



Source: GAO analysis of Washington Scholarship Fund data. 



Simultaneous Growth of Other Choice Options. The Act directs the 
evaluator to determine the effects of the program on District public 
schools, but the opportunities for choice within the public school sector, 
such as charter schools and magnet programs, are far more numerous than 
those offered by OSP and have grown over the same time period, making 
the determination of OSP’s independent effect on public schools, if any. 
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very difficult, if not impossible, to assess. The evaluation design was based 
largely on an assumption that the award of an opportunity scholarship 
would affect student achievement and public schools by expanding school 
choice to include higher performing schools. This assumption, however, 
does not account for the large number of public charter schools, magnet 
schools, and other specialized programs available through the District of 
Columbia as well as scholarships offered by individual private schools or 
other private organizations. Because these opportunities are far more 
numerous than those offered by OSP and expanded in a similar time 
frame, it would be difficult to attribute changes in the behavior of 
traditional schools uniquely to OSP. Nor were participating private schools 
necessarily limited to those that might be higher performing. Moreover, 
the large portion of some schools’ enrollment composed of OSP students 
suggests that, in some instances, program effects, if any, may be more 
profound for the participating schools that the scholarship users attend 
than for the public schools from which the students departed. 

Inability to include some scholarship recipients in the evaluation. In the 
first year of the program. Education and WSF decided to process 
applications in May and hold the placement lottery in August. At that time, 
however, the program was undersubscribed at the kindergarten through 
grade 5 level; that is, the number of eligible applicants from public schools 
who attended kindergarten through grade 5 was less than the number of 
private school openings available. Consequently, Education and WSF gave 
scholarships to approximately all 800 eligible public school applicants in 
these grades, thus eliminating them from the evaluation. In addition, only 
55 eligible students from schools in need of improvement at the middle 
and high school levels applied that year, and Education and WSF decided 
to offer these students scholarships on a non-random basis, thereby 
eliminating these 55 students from the evaluation. Education and WSF 
further reduced the number of students in the cohort 1 evaluation pool by 
awarding scholarships to about 200 students who were already attending 
private schools. Although the Act did not specifically prohibit scholarship 
offers to students in private schools, the evaluation was designed to make 
comparisons among students who had attended only District public 
schools.^** Therefore, the contractor excluded them from the evaluation so 
as not to reduce the usefulness of the evaluation findings. As a result of 



^^Education and WSF considered eligible students who would be old enough to attend 
kindergarten when scholarships were available as public school students. They did not 
receive any additional priority in the placement lotteries, regardless of the designation of 
the public school they would have normally attended. 
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these decisions, of the 2,454 students who were offered scholarships in the 
first 2 years of the program, only l,387--less than 60 percent— could be 
included in the evaluation. (See table 8.) 



Table 8: Eligible Applicants Offered and Not Offered Schoiarships and Numbers Inciuded in and Excluded from the 
Evaluation 




Eiigibie applicants 




Number included in evaiuation 


Number exciuded from evaiuation 


Cohort 


Offered 

scholarship 


Not offered 
schoiarship 


Total 


Offered 

schoiarship 

(treatment) 


Not offered 
schoiarship 
(controi) 


Totai 


Offered 

schoiarship 


Not offered 
scholarship 


Totai 


1 


1,366 


482 


1,848 


299 


193 


492 


1,067 


289 


1,356 


2 “ 


1,088 


728 


1,816 


1,088 


728 


1,816 


0 


0 


0 


Totai 


2,454 


1,210 


3,664 


1,387 


921 


2,308 


1,067 


289 


1,356 



Source: GAO analysis of Westat data. 

“Numbers for cohort 2 exclude eligible private school students because the evaluation contractor 
assigned them a zero percent probability of receiving a scholarship offer. 



Because of the limited size of the treatment and control groups, the 
evaluation contractor combined the cohort 1 and cohort 2 treatment and 
control groups to make comparisons after year one. Future evaluations 
will compare outcomes for students in these cohorts after they have 
participated in the program for 2 and for 3 years. 

Crossover. Crossover between treatment and control groups can reduce 
the effectiveness of randomization and can make comparisons between 
treatment and control groups difficult to interpret and generalize. While 
scholarship offers made to applicants in the treatment group were not 
withdrawn even if they failed to find a private school placement, 
applicants assigned to the control group sometimes found other means to 
attend private school. In fact, 15 percent of students in the control group 
who provided math scores were enrolled in private schools. The 
evaluation contractor used statistical adjustments to bound its estimates 
of treatment effects for both control group students who attended private 
schools and those who could not or did not use their scholarships. 
However, the contractor did not consider potential selection effects as a 
result of the inability of students to find an acceptable placement as 
opposed to declining the use of a scholarship for some other reason. While 
some students in the treatment group did not use their scholarships or did 
not do so consistently and some students in the treatment group who 
wanted to use their scholarships could not because no school, or no 
school of their choice had openings or would accept them, the number 



Page 42 



GAO-08-9 District of Columbia Opportunity Scholarship Program 






who did not use scholarships specifically because they were unable to find 
acceptable placements is unknown. 

Concentration of scholarship students in a subset of schools. Because 
private schools — which differ greatly in motivation, capacity, and ability to 
accept OSP students — determine the number of OSP students to accept, 
large numbers of scholarship users have been clustered within a small 
subset of private schools. Additionally, OSP students constituted 
60 percent or more of total reported enrollment in three participating 
schools in school year 2006-2007. The evaluation contractor did consider 
the potential influence of clustering of students within families and in 
baseline schools prior to OSP participation on variance estimates, which 
are used in determining the effects of scholarship offers and scholarship 
use. However, the evaluation does not discuss the potential impact of 
clustering large percentages of OSP students within relatively few schools. 
Such clustering of evaluation participants within a small number of 
schools has the potential to confound program and school-level effects 
and may distort differences between the comparison groups unless 
appropriate statistical methodologies are employed to disentangle these 
effects when analyzing the effect of receiving a treatment. 

Combining scholarship offers with other supports. In addition to offering 
scholarships, the program provides scholarship students with additional 
individual supports, which may include case management and parent 
empowerment services from the grantee, summer school, tutoring, 
remedial classes, before-and-after school care, and mentoring. Throughout 
their participation in the program, OSP students receive different types 
and combinations of these services that were not readily available to 
students in the control group, bringing into question the comparability of 
students in the treatment and control groups and raising problems in 
generalizing the study’s findings to circumstances in which a scholarship is 
the only treatment offered. 



Some Required The Act required the contractor to compare the safety of schools attended 

Comparisons Relied on by participants in the program and the schools attended by students who 
Limited Data participate in the program and to evaluate the success of the 

program in expanding choice options for parents. In its evaluation of 
effects after year one, the contractor relied exclusively on self-reported 
data to compare the safety of schools attended by the two groups, using 
survey responses to compare perceptions of school safety among parents 
of students offered scholarships and students offered scholarships with 
the perceptions of those who were not and their parents. While 
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perceptions may be a valid measure, they do not necessarily reflect the 
safety risks students face at a given school, and parent and student 
perceptions sometimes diverged. While not specifically required by the 
Act, the contractor also used surveys to compare school satisfaction of (1) 
parents of students offered scholarships and (2) students offered 
scholarships, with (3) school satisfaction of parents of students not 
offered scholarships and (4) students not offered scholarships. In the 
evaluation report, the contractor cast satisfaction as “an indicator of the 
success of the program in expanding options for parents.” 



Conclusions 



The scholarship program administered by WSF has provided low-income 
families in the District an additional option to enroll their children in 
private schools. However, to maintain a program capable of using public 
funds for their intended purpose — that of providing increased 
opportunities to low-income parents to send their children, particularly 
those attending schools designated as in need of improvement, to private 
schools — the agency or organization with responsibility for operating a 
school choice program such as OSP needs a strong financial 
accountability infrastructure that incorporates a system of internal 
control. 

During the period of our review, through year three of the program, WSF 
struggled to build an accountability infrastructure that could assure 
Congress, families of participating students, and the public that its funds 
were used effectively and in compliance with laws and regulations. Strong 
accountability and internal controls over a scholarship program, such as 
OSP, are critical for ensuring that funds are used for their intended 
purposes, and to maintain fiscal viability and credibility. Overall financial 
management policies and procedures and the underlying systems need to 
provide assurance that federal fimds are being used for the purposes 
intended and that funds are safeguarded against loss from error, abuse, 
and fraud. 

Without complete and accurate information about schools’ performance 
and other quality indicators, parents cannot make well-informed choices 
among schools for their child. The grantee did not provide parents 
information about the achievement levels of all students in participating 
private schools and other indicators of school quality and, for some 
schools, provided inaccurate information about teacher qualifications and 
tuition levels. As a result, parents might have used opportunity 
scholarships to place their children in private schools that were less 
successful in raising achievement levels than the public schools their 
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children previously attended. In other cases, parents might have rejected 
some private schools because they were given inaccurate information 
about the schools or, in the case of the few schools that do not 
customarily charge tuition, applied for a scholarship in the belief that their 
children needed scholarships to attend these schools. 

In the absence of an accountability mechanism to ensure that participating 
schools are operating lawfully in the District, students were placed at risk 
of attending facilities that did not meet basic health and safety 
requirements. Because OSP is a federal program that had the support of 
the District of Columbia’s government, parents may have incorrectly 
assumed that Education and the city were overseeing the program and 
making sure participating schools met such standards. As a consequence, 
parents may have been less inclined to investigate the status of these 
schools. 

Finally, implementation of educational evaluations involving random 
assignment often requires substantial oversight and structure. Without 
such oversight and structure, the ability to draw valid conclusions about 
the program to improve educational policy can be compromised by the 
zeal to ensure program participants’ success or to maximize access to a 
treatment that program operators and applicants may believe is beneficial 
even in the absence of empirical evidence. Indeed, throughout the short 
history of OSP, decisions made to advance program goals, such as 
maximizing the number of scholarships awarded in the initial year and 
providing a wide range of academic and social supports to scholarship 
users, have undermined the goals of the evaluation to produce meaningful 
findings and the ability of Congress to use these findings for decisions 
about other programs. 



Recommendations for 
Executive Action 



• establish and implement detailed policies and procedures to improve 
financial controls over OSP grant funds, including specific 
requirements for the process of approving scholarship payments and 
documentation of the process; 



Given the importance of using funds appropriately to further program 
objectives and help ensure that schools are safe and provide sound 
educational experiences, parents are given accurate information about 
schools their children may attend, and program oversight is sufficient, we 
are making the following recommendations. Specifically, we recommend 
that the Secretary of Education direct WSF, the grantee, to 
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• establish compensating controls, such as supervisory review, to reduce 
the risk of fraud in situations where segregation of duties is not 
possible due to the size limitations of OSP’s staff so that no one 
employee can authorize, process, review, and have access to the funds 
relating to OSP; 

• continue its efforts to implement an integrated financial management 
system to facilitate processing and recording of scholarship payments 
and overall financial reporting; 

• develop and implement procedures for conducting site visits, including 
that site visit reports be prepared and contain information on the 
overall financial stability of the school; and 

• develop procedures to ensure that accurate information is provided to 
parents before a school is chosen about the summary achievement data 
of students, teacher qualifications, and tuition levels, and that schools 
make such information available on an annual basis to parents of 
enrolled students. 

We also recommend that the Secretary of Education collaborate with the 
Mayor of the District of Columbia to ensure participating schools are in 
compliance with ah relevant District of Columbia education and safety 
requirements, including school accreditation and health, safety, and fire 
code requirements, and receive approval to operate in the District. 

Further, we recommend that the Mayor direct the Office of the State 
Superintendent of Schools to collaborate with the Board of Education to 
develop and implement procedures for ensuring that private schools in the 
District meet applicable District requirements and to actively participate in 
the oversight of OSP. Finally, we recommend that, in planning for future 
programs for which Congress has required an evaluation, the Secretary of 
Education should take steps to make certain the program to be evaluated 
is overseen to ensure it is implemented in a manner consistent with the 
evaluation design. 



Agency Comments 
and Our Evaluation 



At our invitation, the Office of the Mayor of the District of Columbia (the 
District), Education, and WSF provided written comments on a draft of 
this report, which are reproduced in appendixes 111, IV, and V, 
respectively, and summarized below. WSF also provided technical 
comments, which we incorporated when appropriate. In general, all three 
entities concurred with our recommendations, but Education and WSF 
disagreed with many of our findings and interpretations. 

The District agreed with our findings and recommendations. It said the 
report’s findings have been extremely helpful to Mayor Fenty’s 



Page 46 



GAO-08-9 District of Columbia Opportunity Scholarship Program 




administration as it developed its plan for moving forward to fulfill its 
responsibilities under the MOU with Education, particularly on the steps 
the District should take to ensure that parents receive accurate and 
complete information and that schools receiving funds under the program 
meet certain basic health, safety, and instructional requirements. The 
District’s comments include specific ongoing or planned steps to ensure 
that WSF complies with the requirements of federal law and applicable 
local statutes and regulations. In addition, the Deputy Mayor for 
Education has directed the District’s Department of Consumer and 
Regulatory Affairs to conduct inspections at the schools identified as 
having missing or inapplicable certificates of occupancy and is developing 
an accelerated schedule for reviewing the regulatory status of all schools 
receiving funds under OSP. In its comments, WSF supplied additional 
information about one of the several schools DCRA identified as lacking a 
certificate of occupancy reflecting operation of a private school. After 
verifying this information with DCRA, we changed the draft to reflect it. 

With one exception. Education generally agreed with our 
recommendations and stated that it would use the information in our 
report to continue to improve its oversight of the program and its 
coordination with the District. With respect to our recommendation that, 
in planning for future programs for which Congress has required an 
evaluation. Education ensure that the program is implemented in a manner 
consistent with the evaluation design. Education did not express 
agreement or disagreement. Rather, Education’s comments focus on the 
OSP evaluation, indicating that we overemphasized the evaluation’s 
challenges resulting from program implementation decisions and other 
factors and therefore underestimated the evaluation’s utility. Specifically, 
Education stated that the inability to compare OSP students to all students 
does not undermine the more important analysis of program impact. While 
Education adopted a strong methodology for its analysis of impact, as 
required by statute. Education’s comment is not responsive to our point 
that its evaluation was unable to provide a comparison between students 
offered scholarships and those in the same grades in District schools, as 
the statute mandated. Also, Education wrote that the inability to include 
some early scholarship recipients in the evaluation is not a major issue but 
also noted that the Department rejected the notion of excluding these 
early recipients altogether, “so that there would be sufficient samples of 
students to allow impacts to be estimated for subgroups of students..’’ 

We maintain excluding about half of students offered scholarships from 
the evaluation will affect the ability of the evaluation to make strong 
comparisons. Education also wrote that other supports provided by WSF 
were not organized and were available to both the treatment and control 
groups. We disagree. According to WSF, it offered case management 
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services, conducted parent empowerment sessions, found and worked 
with community organizations that provide mentoring and tutoring, and, 
on a case by case basis, funded private schools to provide enrichment and 
remedial services to support OSP students. We continue to believe that 
the challenges we discuss have complicated the interpretation of 
evaluation findings, limit generalizability, and merit close examination 
when planning for the evaluation of future programs. 

Education’s comments also expressed disagreement with many of our 
findings. Education believes we mistakenly concluded that WSF’s practice 
of paying for tuition scholarships to schools that normally receive 
donations to cover tuition violates the Act and WSF’s policies. We 
disagree. As discussed in the report and as expressly confirmed in WSF’s 
comments on the draft report, children who attend the three scholarship- 
only schools pay no tuition. Section 307(a) (1) of the D.C. School Choice 
Incentive Act states that, to be reimbursed under OSP, the tuition charged 
by schools may not “exceed the amount of tuition or fees that the school 
customarily charges to students who do not participate in the program.” 
(emphasis added) As section 307(a) (1) makes clear, the amount of 
allowable tuition reimbursements depends upon the amount of tuition the 
schools charge to students, not what costs schools incur or what other 
funding mechanisms, such as donated scholarships, they use to cover 
those costs. 

In addition. Education stated its belief that the report does not present a 
complete and balanced picture with respect to (1) the extent to which OSP 
students previously attended schools in need of improvement; (2) the 
establishment of OSP in time for the 2004-2005 school year; (3) responses 
from parents and demand for scholarships; and (4) participating schools’ 
legal independence in areas such as hiring and establishing teacher 
qualifications. With respect to the extent to which OSP students 
previously attended schools in need of improvement. Education wrote that 
it would have been more accurate to focus on the percentage of students 
from schools that received an improvement designation during the 
student’s first year in OSP. We disagree. Scholarship lottery organizers 
could not be expected to give priority to students from schools that had 
not yet been designated as in need of improvement. Moreover, 
regardless of which year is considered, the percentage of students from 



We used the lists of schools identified as in need of improvement at the time the lotteries 
were held to compare OSP scholarship recipients to District public and charter school 
students— the pool from which they were to have been recruited-in terms of the 
percentage of students from schools designated in need of improvement. 
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schools in need of improvement was consistently smaller among 
scholarship recipients than among the general population of District 
students from whom scholarship recipients would have been recruited. 
With respect to the establishment of OSP in time for the 2004-2005 school 
year, we included a timeline delineating the key events in implementing 
this program and discussed the short time available to implement the 
program in the draft we provided Education to review and in this report. 
Education indicates that GAO should have included information about 
positive responses from parents of scholarship students enrolled in private 
schools. A balanced picture of parental satisfaction would also include 
views of parents whose children no longer participate or comparisons to 
school satisfaction among parents who elected not to participate, but 
collecting such data would have been tangential to our objectives 
regarding mechanisms to account for program fimds, results of efforts to 
meet recruiting targets, and review of the evaluation. With respect to 
demand for scholarships, as discussed in appendix I, we attempted to 
work with WSF’s data on program applicants and found these data 
imreliable. Finally, Education commented that participating schools were 
allowed to use their traditional independence and that the intent of the law 
was to maximize participation of private schools in the program. We agree 
but, as noted above, maintain that participating schools are required to be 
in compliance with District education and safety requirements. We believe 
we appropriately addressed each of these areas and, therefore, have not 
made any changes based on Education’s comments. 

Education also expressed dissatisfaction with our exit briefing and asserts 
that GAO refused to brief the Department on its proposed findings and 
recommendations prior to sending the Department the draft report. 
However, at a July 10, 2007, exit conference, the meeting that Education 
mentions in its comments, we presented our findings. The findings we 
present at exit conferences are preliminary in that they could change on 
the basis of information we receive at the exit conference as well as 
information contained in official agency comments. We also held exit 
conferences with WSF and the Office of the Mayor. We considered all 
information provided by Education, WSF, and the Office of the Mayor at 
and subsequent to the exit conferences. Education also noted its concern 
about the inappropriate disclosure to the news media of the report draft 
we had provided to Education, WSF, and the Office of the Mayor for 
comment. We share this concern. While we have policies and procedures 
designed to help prevent premature disclosure, we cannot ensure that 
drafts will not be prematurely released once they leave GAO’s control. 

Education provided further comments: 
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• Education wrote that the concentration of scholarship recipients in 
a subset of participating private schools does not interfere with 
accurate estimation of the program impact. GAO believes it is 
essential to investigate whether OSP’s effects are generalizable or 
limited to a small subset of participating schools, and we note that 
the evaluation contractor’s report did not explicitly discuss the 
potential effects of clustering in private schools. 

• Education requested that GAO investigate the improper disclosure 
of the report immediately. GAO is examining the circumstances 
related to this specific instance of premature release. In this 
regard, we would welcome Education’s assistance by requesting 
that its Inspector General review the Department’s policies and 
procedures for handling draft GAO reports in its control. 

• Education noted that District requirements do not require teaching 
staff to hold District teaching certificates. We agree and have not 
asserted District teaching certificates were required. 

• Education wrote that WSF has made significant progress in 
establishing formal policies and procedures for improved internal 
controls and integrating its financial management systems. GAO 
had included this information in the draft report. However, as 
noted by GAO and WSF, efforts are still underway to integrate the 
scholarship payment processing with the financial system, which 
was scheduled to be launched on November 1, 2007. WSF expects 
to finalize its policies and procedures in January 2008. 

• Education stated in its comment letter that since WSF’s receipt of 
the federal grant funds, WSF’s annual independent financial A-133 
audits have identified no reportable conditions or material 
weaknesses. However, WSF’s auditors did report an instance of 
noncompliance with OSP requirements based on the finding that 
WSF had awarded scholarships to four, out of 40 students tested, 
from households that did not meet the income eligibility 
requirements. The auditor added that the effect of this condition 
was that WSF expended federal funds on ineligible recipients. In 
response to this finding, WSF stated that it terminated the contract 
with the outside vendor and brought the eligibility verification 
process in-house and also developed numerous additional 
controls. 
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In WSF’s comments, WSF acknowledged that some of GAO’s 
recommendations, findings, and observations were valid and useful and 
had taken or will take action to address some of them but disagreed with 
many of our observations and findings. As we noted earlier in our 
response to Education, we believe the findings and interpretations are 
accurate as stated in the draft, and therefore we have not made changes 
based on WSF’s comments. WSF disagreed with our interpretation of the 
D.C. School Choice Incentive Act with respect to making payments to 
schools that do not customarily charge tuition to their students, WSF’s 
success in meeting recruiting priorities, and the extent to which it had 
provided complete information to parents. WSF also stated that it 
“vigorously disagrees with our conclusion that there must be formal 
‘academic support activities’ in order for before-and-after school programs 
to contribute to student success.” We disagree that our draft report makes 
this conclusion. We do, however, conclude that we could not determine 
based on the documentation provided by WSF that WSF actually verifies 
that before-and-after school care programs are tied to the student’s 
academic program and part of customary fees charged by the school, 
requirements that are Education and WSF’s criteria for determining 
whether a fee is allowable. 

WSF also indicated that GAO erred in analyzing the participation of 
students from schools in need of improvement, skewing the result toward 
finding underrepresentation, by not assessing students by grade level. 
GAO's analysis covered participation by students from schools in need of 
improvement across all the grades the program was intended to serve. 

The draft report we previously provided to them discussed the role of 
grade level in an applicant's effective probability of receiving a 
scholarship. 

WSF officials also stated that our report focused primarily on matters 
bearing little on the financial viability and effectiveness of the OSP; 
however, evaluation of the financial viability of the OSP was not within the 
scope of this engagement. As agreed with the requesters, GAO focused on 
the identification and assessment of accountability mechanisms over 
appropriated funding to implement the D.C. School Choice Incentive Act. 
Implementation of effective internal control is key to achieving 
accountability over grant programs, such as the OSP. We reiterate that 
while WSF has taken actions to improve accountability, such as 
integrating its student database with its online billing system, as of the end 
of our field work, its operations were still hampered by the lack of 
integration between its scholarship payment processing and its financial 
accounting system and the related impact on accountability over cash. We 
note that the preparation of bank reconciliations is a key cash 
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management control. We are pleased that WSF has begun a process to 
redo all bank reconciliations starting with 2004. We note further that 
efforts to implement a new financial system began during the time of our 
review, and according to WSF officials, they now expect the new system to 
be launched in November 2007 and to result in an increased level of 
efficiency and accountability. 

While WSF believes that the deficiencies noted in this report are minor in 
nature, we stress that they are indicative of potential problems and if not 
addressed could possibly have a material, detrimental effect on WSF’s 
accountability over federal funds, especially when combined with the 
accounting systems and cash reconciliation weaknesses that existed. WSF 
hypothetically submits that the incidence of error is inflated by GAO 
counting a missing signature on one fee form as 20 errors if the fee were 
charged to 20 students’ scholarships. Our test results show that the 39 
students (78% of the sample) referenced in our report attended 19 d iff erent 
schools participating in the program and that the respective fee forms for 
those schools were missing an authorizing signature. 

In its comments, referring to its oversight visits to the schools in the 
program, WSF also stated that “GAO puzzlingly reported that WSF did not 
provide evidence of those visits.” While WSF provided us a list of the 42 
schools that were visited, WSF was only able to provide one completed 
report documenting a visit to one of these schools. 

WSF commented, with reference to what it understood the rationale to be 
for GAO’s audit, that GAO’s draft report cites absolutely no evidence that 
federal OSP funds have been spent for anything other than genuine 
educational purposes at any time during the three-plus years of OSP’s 
operation. As we noted in the report, GAO identified instances in which 
payments were made to some OSP schools for before-and-after care 
services, and it was not clear, based on the documentation provided by 
WSF, whether these services were tied to educational activities. 

In addition to comments that were also made by Education, WSF wrote 
that GAO cited no evidence that OSP families have not received the 
educational services that they sought through participation in the OSP. 

The direct investigation of the extent OSP families received the services 
that they sought was outside the scope of our work. We did determine, 
however, that some families declined the use of scholarships and that 
others did not remain in the program. For example, in year one of the 
program, 32 percent of students offered scholarships did not use them, 
and of the 68 percent that did, 31 percent did not use them in all the years 
the scholarships were available to them. 
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We are sending copies of this report to the Secretary of Education, the 
Mayor of the District of Columbia, and the President of the Washington 
Scholarship Fund, appropriate congressional committees, and others who 
are interested. We will also make copies available to others upon request. 
In addition, the report will be available at no charge on GAO’s Web site at 
http://www.gao.gov. 

If you or your staff have any questions or wish to discuss this material 
further, please call Cornelia M. Ashby at (202) 512-7215 or Jeanette M. 
Franzel at (202) 512-9471. 

Cornelia M. Ashby 

Director, Education, Workforce, and Income Security Issues 
Jeanette M. Franzel 

Director, Financial Management and Assurance 
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To address our research objectives, we focused our efforts on assessing 
three areas: (1) the accountability mechanisms that Washington 
Scholarship Fund (WSF) has put in place to govern the use of funds, 

(2) the results of WSF’s efforts to meet the program’s recruiting priorities 
and eligibility requirements and inform parents of their choices, and 

(3) the extent to which the evaluation reflects statutory requirements and 
the implementation of the program supported the detection of useful and 
generalizable findings. In performing our work to address these objectives, 
we interviewed officials from WSF, the Department of Education’s Office 
of Innovation and Improvement and Institute of Education Sciences, and 
District government officials. In doing our work, we requested and 
received program, demographic, testing, and funding data from both WSF 
and Westat. We encountered data reliability problems that affected the 
achievement of these objectives. These problems and the resulting audit 
work we performed are described below. 



Assessing the 
Expenditure and 
Accountability of 
Grant Funds 



In assessing the expenditure and accountability of grant funds, we met 
with grantee and Education officials, reviewed key documents to identify 
expenditures and key internal controls, and then tested the application of 
those controls for school year 2005-2006 through sampling. To identify the 
changes in the grantee’s payment processes from 2004 and key internal 
controls, we met with the grantee staff and reviewed their procedures. 

Due to limitations in the grantee’s financial accounting software, we used 
the Excel spreadsheet for school year 2005-2006 as the population of 
school scholarship payments from which to draw our sample of 
50 students. Before selecting our sample, we were unable to determine 
that the student scholarship payments for school year 2005-2006 were 
complete and reliable because the total of the payments on the 
spreadsheet could not be directly traced to the grantee’s financial 
accounting software or the total of funds drawn down from Education. 
Therefore, we pulled a random sample of students for whom scholarship 
payments were made for school year 2005-2006. We tested all OSP 
scholarship payments made during school year 2005-2006 for each of the 
50 students selected randomly to determine whether key internal controls 
were being properly implemented for those transactions. 
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Assessing 
Demographic and 
Program Data 



In the process of assessing data reliability, we found that the data were 
sufficiently reliable to address our objectives. However, reliability 
problems with the WSF student demographic and program data system, 
limited the analyses that we could perform. Data problems included the 
following: 

• The data were not integrated and different data sets were maintained 
on d iff erent spreadsheets. 

• The student data lacked unique and uniform identification numbers 
that would have allowed the efficient linkage of key data sets from 
different data and, therefore, made it necessary to link the students’ 
data by first and last names. 

• Many data entry errors including misspellings of names and duplicated 
entries. 

• Substantial missing data for such important data fields as race, grade 
level, and date of birth made the fields insufficiently populated for 
analyses; for example, 32 percent of data identifying gender, 34 percent 
identifying birth date, and 32 percent identifying race were missing. 

As a result, we were limited in our ability to describe characteristics of 
students in the program. We were also limited in our ability to link student 
demographic and financial eligibility data with payment data. We reviewed 
WSF processes for collecting financial eligibility information and the 
household income data reported by families of students offered 
scholarships but we did not test those data as to accuracy. Although we 
determined that students in the program for whom we had data generally 
met financial eligibility requirements, we were unable to positively ensure 
that all students who were receiving Opportunity Scholarship Program 
(OSP) funds had been determined eligible by WSF due to missing data 
payment. Using WSF’s payment records, we identified 50 students who 
WSF indicated had received scholarships and were attending private 
schools, but we could not electronically locate eligibility data for these 
students. As a result, we could not verify whether these students were 
eligible. We subsequently sent these names to WSF, which provided 
alternative spellings for the students’ names that enabled us to reduce the 
50 nonmatching records to 18. Although these students represent a small 
percentage of scholarship users, the absence of their financial eligibility 
data represented a considerable weakness in the data system. 

Although we could correct some data errors through manual checking, we 
were unable to correct and eliminate apparently duplicate records for 
students who had applied for the program. Due to these duplicate records, 
we were unable to determine the number of students who applied for the 
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OSP and, of those, the number and percentage who were determined to be 
eligible. We attempted to work with the WSF to identify which records 
were duplicates in the applicant data, and which were valid applicants, but 
due to data problems we were unable to make these identifications. 
Instead, we could only report on the number entered into the lottery and 
whether they received scholarship awards or not. 

We used data from WSF to create a database to summarize characteristics 
of participating private schools. We used data provided by the District of 
Columbia Public Schools to identify schools designated as in need of 
improvement and calculate the numbers and percentages of students who 
attended these schools by year. 

Data limitations prevented us from using computerized methods to match 
information on WSF data sets with the evaluator contractor’s database. We 
attempted to match data from WSF with data collected by the evaluation 
contractor as a reliability check. However, because the WSF and the 
evaluation contractor did not develop common identifiers for designating 
participants in the program, we were unable to match students in an 
automated fashion, and therefore had to match records using student first 
and last name. Despite intensive efforts, including manually making 
spelling changes to over 492 student names, the volume of data 
mismatches would not allow us to develop a systematic method to match 
the records of WSF and the evaluation contractor. 

The evaluation contractor matched its data to WSF records by attempting 
to match on a variety of including first and last name and birth date; 
developed a crosswalk to link by household number; and by manually 
matching records. The contractor also used a computer application that 
matched names to other names by phonetic spelling; that is, names were 
matched with other names that sounded similar, allowing it to match most 
of the data. We determined that this method was not sufficiently reliable 
for our purposes because some names had similar phonetic spelling. 

Additionally, we found large discrepancies between the numbers of 
applicants and eligible students included on the evaluation contractor’s 
database and in the WSF files. This further decreased our confidence in 
the use of these data for this population. 
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Assessing Strategies 
to Recruit Families 
and Private Schools 
and Inform Parents 
about Their Choices 



To assess how the grantee was meeting the program’s recruiting and 
eligibility objectives and informing parents about their choices, we 
interviewed knowledgeable officials and examined program 
documentation, especially relating to recruitment and selection of both 
students and private schools. Apart from reviewing publicly available 
information about schools, we did not evaluate schools or their 
performance. To determine the number of OSP students who had attended 
schools in need of improvement, we identified the type and No Child Left 
Behind status of schools previously attended by students in cohorts 1, 2, 
and 3, and quantified the number of students who had attended District of 
Columbia public schools in need of improvement for each year that the 
program operated. To determine whether all participating private schools 
met the requirements to operate lawfully in the District, we selected a non- 
probability sample of 18 schools, using various criteria such as whether or 
not schools had registered with the District’s Department of Consumer 
and Regulatory Affairs and whether or not schools were accredited by an 
agency recognized by the District To determine the validity of information 
in the directories published by WSF in order to see if information provided 
parents was accurate, we compared directory information with 
information available from other sources. In reviewing the directory 
information, we became aware of discrepancies in information reported in 
English and Spanish versions of the directory, and information provided 
across different school years. We found the information across sources, 
across years, and between the English and Spanish version was 
inconsistent. 
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Assessing the Extent 
to Which the 
Evaluation Reflects 
Statutory 

Requirements and the 
Implementation of the 
Program Supported 
the Detection of 
Useful and 
Generalizable 
Findings 



To assess the extent to which the evaluation reflects the requirements in 
the Act and to which the implementation of the program supported the 
detection of useful and generalizable findings, we met with Education 
officials from the Institute of Education Sciences and examined 
documents including the contractor’s participation reports for program 
year one and program year two, the analysis plan it developed for its 
evaluation of impacts after 1 year of program participation, and the 
Evaluation of the D.C. Opportunity Scholarship Program: Impacts After 
One Year, released in June 2007. We also reviewed contractual 
documentation. To assess the usefulness of evaluation findings, we 
examined program attrition, the extent of missing test data, and the 
statistical methodology the contractor used to analyze comparisons 
between students offered scholarships and students not offered 
scholarships, and, to a lesser extent, between students in these groups 
who used their scholarships and those who remained in public schools. 



Page 58 



GAO-08-9 District of Columbia Opportunity Scholarship Program 




Appendix II: Laws Authorizing Related 


Voucher Programs 






Table 9: Analysis of Laws Authorizing Voucher Programs in the District of Columbia and the State of Ohio 


Analysis of state laws on 
private school vouchers 


District of Columbia 


State of Ohio 


Name of program 


D.C. School Choice Incentive Program 


Educational Choice Scholarship (EdChoice) 
Pilot Program 


Year enacted 


2004 


2006 


Description and eligibility 
requirements 


The Opportunity Schoiarship Program, aiso known 
as the D.C. School Choice Incentive Program, 
provides scholarships to students for attendance at 
private schoois in the District of Columbia, it is 
federaliy funded. To be eiigibie, students must be 
from families who reside in the District and whose 
househoid income does not exceed 185 percent of 
the federai poverty levei. Schoiarship recipients who 
received their first scholarship in 2004-2005 or 
2005-2006 schooi year may retain their schoiarships 
if their household income does not exceed 300 
percent of the poverty level. If the number of new 
scholarships in any year is less than the number of 
eligible applicants, selection of recipients follows a 
lottery method. Priority is given to students attending 
schools identified for improvement, corrective action, 
or restructuring under Title I of the No Child Left 
Behind Act of 2001 . 


Students who attend or will be entering Chio 
public schools that have been designated by 
the state as “Academic Watch” or “Academic 
Emergency” for 2 of the last 3 years are eligible 
to receive scholarships to attend the 
participating private school of their choice. 
Students currently enrolled in charter schools 
but who would otherwise be assigned to 
schools in these categories are also eligible. 
Students in the Cleveland Municipal School 
District are not eligible to participate, as the 
state offers a separate scholarship program for 
these students. Scholarships are not available 
to students currently enrolled in a private non- 
public school or who are home-schooled. 
Eligible students must first be accepted at a 
participating private school for the next school 
year before applying for an EdChoice 
scholarship. 


Currentiy implemented 


Yes 


Yes 


Participating students 


1,819 


2,785 


Geographic area 


D.C. 


State of Chio, except Cleveland 


Limit on number of students 


Limited by available funding 


Limit of 14,000; if number exceeds 
14,000, priority will be given to students who 
already received scholarships and with family 
incomes at or below 200 percent of the poverty 
level, and students will be elected by lot to 
receive remaining scholarships. 


Grades 


K-12 


K-12 


Average or maximum amount 
of assistance 


$7,500 


$5,000 


Criteria for determining 
amount of assistance 


The annual scholarship amount is $7,500 or the 
participating private school’s tuition, fees, and any 
transportation costs, whichever is less. 


The EdChoice scholarship amount is currently 
set at $4,250 for elementary school students 
(grades K-8) and $5,000 for high school 
students or the private school’s tuition amount, 
whichever is lower. The scholarship amount 
will increase slightly each year. 


Types of schoois ailowed to 
participate 


Any private elementary or secondary school within 
DC, including religious schools. 


Any chartered nonpublic school that meets the 
state’s requirements. 
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Analysis of state laws on 
private school vouchers 


District of Coiumbia 


State of Ohio 


Testing or evaluation criteria 


As mandated by law, the program is evaluated 
rigorously on an annual basis by an independent 
research organization. Evaluations address the 
academic achievement and the retention, dropout, 
and college admissions rates of scholarship 
recipients, in comparison both to students who 
remain in D.C. public schools and to students who 
applied for but did not receive scholarships. 
Evaluations also examine the following: the success 
of the program in expanding educational options for 
parents; the reasons why parents choose to have 
their children participate in the program; the impact 
of the program on students and public schools in the 
District; and the safety of the schools attended by 
scholarship students, in comparison to other D.C. 
schools. 


Private schools are required to administer state 
achievement tests to scholarship students. 


Accreditation of private 
schools 


Not specified 


A nonpublic school must hold a valid state 
charter and comply with the operating 
standards for Ohio’s schools and agree to 
register with the Ohio Department of Education 
and comply with the rules of the program, 
including administering state achievement 
tests. 


School admission 
requirements 


Not specified, but does state that students must 
abide by rules of the school applicable to all 
students. 


Not specified, but students must gain 
admittance to the eligible private school before 
applying for the scholarship. 


Discrimination provision 


Certain exemptions from nondiscrimination 
requirements are given to participating schools with 
a religious affiliation. Participating private schools 
cannot discriminate on the basis of race, color, 
national origin, religion, or sex (except for single-sex 
schools). 


Not specified 


Religious “opt out” close 


Not specified 


Not specified 


Financial audit requirements 


Administrative entity is responsible for ensuring that 
participating schools are financially responsible. 


Not specified 


Transportation 


Funds may be used for transportation expenses. 


Students enrolled in nonpublic schools may be 
entitled to pupil transportation services from 
their public school district of residence. The 
nonpublic school must be within a 30-minute 
bus ride from the public school during the 
school day. In certain cases, a public school 
district may declare a student “impractical to 
transport” and instead provide limited 
reimbursement payment to the parent. 


Authorizing statute 


D.C. Code §38-1851.01 — D.C. Code § 38-1851.11 


Ohio Revised Statues. Sections 3310.01- 
3310.17 
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Analysis of state laws on 
private school vouchers 


District of Coiumbia 


State of Ohio 


Legislative history 


On January 23, 2004, President Bush signed the 
program into law via the D.C. School Choice 
Incentive Act of 2003, which was included in the 
Consolidated Appropriations Act of 2004. In 2006, 
legislation passed that raised the household income 
eligibility renewal limit from 200 to 300 percent of the 
federal poverty level for students who received their 
first scholarship in 2004-2005 and 2005-2006 school 
years. The program was first implemented in the 
2004-2005 year and is the first of its kind at the 
federal level. 


Cn June 30, 2005, the Educational Choice 
Scholarship Pilot Program was signed into law 
as part of an omnibus education bill. Cn 
March 30, 2006, a new omnibus education bill 
was passed that expanded eligibility for 
scholarships under the program to students in 
schools in “academic emergency” or “academic 
watch” for the 3 previous years; the previous 
law limited eligibility to students only in schools 
in “academic emergency,” the lowest category 
in the school rating system. Cn March 30, 

2007, eligibility for scholarships was further 
extended to students in schools in these 
categories for 2 of the previous 3 years. 


Year enacted 







Source: GAO Analysis of State Laws and U.S. Department of Education, Education Options in the States: State Programs that Provide 
Financiai Assistance for Attendance at Private Eiementary or Secondary Schoois (Washington, D.C.: Aug. 2007). 
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Table 10: Analysis of Laws Authorizing Voucher Programs in Cleveland, Ohio, and Milwaukee, Wisconsin 


Analysis of State Laws on 
Private School Vouchers 


Cleveland, Ohio 


Wisconsin — Miiwaukee 


Name of program 


Cleveland Scholarship and Tutoring Program 


Milwaukee Parental Choice Program 


Year enacted 


1995 


1990 


Description and eligibility 
requirements 


The Cleveland Scholarship and Tutoring Program 
provides scholarships to students in the Cieveiand 
Municipal School District. The schoiarships are for 
attendance at a quaiified private school within the 
district or at a public school in any district 
surrounding Cleveland. Students in grades K-8 are 
eligible to apply for scholarships. Scholarship 
recipients may retain their scholarships through 
grade 12. If the number of new scholarships in any 
year is less than the number of eligible applicants, 
selection of recipients follows a lottery method. 
Students from low-income families have priority in 
receiving new scholarships. 


The Milwaukee Parental Choice Program 
(MPCP) provides scholarships to students for 
attendance at participating private schools in 
Milwaukee. To be eligible, students must be 
from families who reside in Milwaukee and 
whose household income does not exceed 
175 percent of the federal poverty level. A 
student participating in the program, and whose 
family income increases, may remain in the 
program until the family’s income exceeds 
220 percent of the federal poverty level. The 
maximum participation is 22,500 students. 
Applications for scholarships are submitted 
directly to participating schools. If a participating 
school receives more applications in any year 
than it has seats available, selection of 
recipients follows a lottery method. Siblings of 
current scholarship recipients have priority in 
receiving new scholarships. 


Currentiy implemented 


Yes 


Yes 


Participating students 


5,921 


17,410 


Geographic area 


Cleveland Municipal School District 


City of Milwaukee 


Limit on number of students 


Number of scholarships limited by the amount of 
state-appropriated funds available. 


22,500 


Grades 


K-8 for new applicants or 9-12 for renewal students 


K-12 


Average or maximum amount 
of assistance 


$3,450 


$6,501 
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Analysis of State Laws on 
Private School Vouchers 


Cleveland, Ohio 


Wisconsin — Milwaukee 


Name of program 


Cleveland Scholarship and Tutoring Program 


Milwaukee Parental Choice Program 


Criteria for determining amount 
of assistance 


To attend a participating private school, the annual 
scholarship amount is based on the tuition of the 
school and the family income level of the recipient. 
Currently, the scholarship amount may not exceed 
the approved private school’s tuition or $3,450 for 
students in grades K-12, whichever is less. For 
recipients whose family income level is below 
200 percent of the federal poverty level, the actual 
scholarship award is 90 percent of (up to) the 
maximum amount (i.e., a student from such a family 
and currently in grade 8 may receive a scholarship 
award of at most $3,105). For recipients whose 
family income is at or above 200 percent of the 
poverty level, the scholarship is 75 percent of (up 
to) the maximum amount. Remaining tuition costs 
are to be covered by parents; however, for 
recipients in grades K-8 whose family income level 
is below 200 percent of poverty, participating 
schools must not charge any tuition in excess of the 
remaining 10 percent of the amount, whatever that 
amount may be. 


For the 2006-2007 academic year, the 
scholarship amount is the participating private 
school’s per pupil expenditure or $6,501, 
whichever is less. Participating schools must 
accept the scholarship amount as full payment 
of tuition. 


Types of schools allowed to 
participate 


Any nonpublic chartered schools within the 
boundaries of the Cleveland Municipal School 
District that meets all applicable requirements. 


Any private school within the city 


Testing or evaluation criteria 


Not specified 


The law requires participating private schools to 
administer nationally normed standardized tests 
to scholarship recipients in grades 4, 8, and 10. 


Accreditation of private 
schools 


The school must meet all state minimum standards 
for chartered nonpublic schools in effect on July 1 , 
1992, except that the state superintendent at the 
superintendent’s discretion may register 
nonchartered nonpublic schools meeting the other 
requirements of this division. 


With respect to participating private schools, the 
law requires schools to be accredited from 
among a list of accrediting agencies. 


School admission 
requirements 


Schools are required to give preference to students 
previously enrolled and their siblings, but specifies 
the number of scholarship students for grades K-3 
equals the number that constituted 20 percent of 
the total number of students enrolled in the school 
during the preceding year in such grade. 
Scholarship students are to be randomly selected 
by lottery. Schools must admit students in grades 
who were previously admitted. 


Participating private schools shall accept pupils 
on a random basis, but may give preference to 
siblings of pupils already enrolled or current 
scholarship participants. May only consider 
income and grade level, not academic 
achievement. 
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Analysis of State Laws on 
Private School Vouchers 


Cleveland, Ohio 


Wisconsin — Milwaukee 


Name of program 


Cleveland Scholarship and Tutoring Program 


Milwaukee Parental Choice Program 


Discrimination provision 


Participating private schools may not discriminate 
on the basis of race, religion, or ethnic background, 
and may not promote unlawful behavior or teach 
hatred. 


A school may not use an applicant’s race, 
ethnic background, religion, priority test scores, 
grades, or membership in the church parish 
when making admissions decisions. 
Participating private schools may not 
discriminate on the basis of race, religion, or 
ethnic background, and may not promote 
unlawful behavior or teach hatred. 


Religious “opt out” close 


Not specified 


Students may be excused from religious 
activities at a religious school if their parent or 
guardian submits a written request to the 
teacher and principal of the school. 
Participating schools may not require a 
recipient to participate in religious activities. 


Financial audit requirements 


Not specified 


The Act requires an annual financial audit and 
specifies other extensive accounting and 
financial requirements. 


Transportation 


Must be provided by Cleveland Municipal School 
District. 


The Milwaukee School District may provide 
transportation or pay some of the cost, in 
certain cases. 


Authorizing statute 


ORC Ann. 331 3.974 - ORC Ann. 331 3.99 


Wis. Stat. § 119.23 


Legislative history 


The program was enacted in 1995 and first 
implemented in the 1996-1997 school year. On July 
1 , 2003, the state legislature amended the program 
to allow recipients to retain their scholarships 
through grade 10, as of the 2004-2005 academic 
year, and also raised the maximum scholarship 
amount from $2,500 to $3,000. In June 2005, the 
legislature expanded the grade range of students to 
whom new scholarships are available from grades 
K -3 to K -8; it also allowed scholarship recipients 
to retain their scholarships through grade 12 during 
the 2006-2007 year and raised the maximum 
scholarship amount available to all recipients, 
regardless of grade, to $3,450 for the 2006-2007 
and subsequent years. 


The Milwaukee Parental Choice Program was 
enacted in 1990. In 1995, it was expanded to 
include religious schools. In 2006, the law was 
changed to increase the number of students 
who may participate in the program, by 
increasing the income limit from 175 percent to 
220 percent over the federal poverty level and 
by eliminating the prior year attendance 
requirements (allowing students who move into 
the district to be eligible). 


Year enacted 


1995 


1990 



Source: GAO Analysis of State Laws and U.S. Department of Education, Education Options in the States: State Programs that Provide 
Financial Assistance for Attendance at Private Elementary or Secondary Schools (Washington, D.C.: Aug. 2007). 
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GOVERNMENT OF THE DISTRICT OF COLUMBIA 
Office of the Deputy Mayor for Education 



Victor Reinoso 

Deputy Mayor for Education 



★★★ 



October 22, 2007 



Cornelia M. Ashby, Director 

Education, Workforce, and Income Security Division 
Government Accountability Office 
441 G Street, NW 
Washington, DC 20548 

Dear Ms. Ashby: 

Thank you for sharing your draft report on the District’s school voucher program with me and 
my staff and for providing us the opportunity to comment on it prior to publication. 

As you stated in the report, the District’s federally-funded voucher program was authorized by 
Congress in January 2004 and has been in operation since the 2004-2005 school year. As you 
know, neither I, nor the Mayor, was in a position during the program’s first three years to 
exercise oversight of the program as envisioned in the Memorandum of Understanding (“MOU”) 
between that the U.S. Department of Education and then-Mayor Anthony Williams. 



Having said that, the report’s findings, and the discussions we had while your team was 
conducting its review, have been extremely helpful to us as we developed our plan for how the 
Fenty Administration would fulfill its responsibilities under the MOU moving forward, and, in 
particular, what steps we should take to ensure that parents considering participating in the 
program receive accurate and complete information regarding the schools their child might 
attend and that schools that receive funds under the program meet certain basic health, safety, 
and instructional requirements. 

I would like to take a moment to share with you the steps we already have taken to improve 
oversight of this program: 

• First, on my recommendation. Mayor Fenty delegated his direct oversight responsibihty 
under the MOU to Deborah Gist, the District’s State Superintendent of Education. Ms. 
Gist heads a new, independent state education agency created as part of the Mayor’s 
education reform legislation. Her agency is charged with oversight of all federal 
education funds received by the District and, as such, I believe it is imiquely positioned to 
serve in this role. 
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• Upon receipt of the delegation letter, Ms. Gist contacted the Department of Education to 
request a meeting regarding the problems in program management by the Washington 
Scholarship Fund (“WSF”) and in oversight identified by GAO. A meeting was then 
convened to discuss how the parties should more closely monitor WSF’s compliance with 
statutory requirements relating to student recruitment, targeting of students attending 
schools “in need of improvement,” tuition and fees, and program evaluation' in the 
future. 

• Because the original MOU between the Department and the District did not contain any 
details regarding coordination or division of their oversight role. Ms. Gist has since 
recommended to the Department that the parties enter into a follow-on agreement that 
clearly assigns certain activities to her agency and others to the Department. 

• In addition, Ms. Gist and her staff met with WSF to explore ways that the District can 
assist that organization in complying with the requirements of the federal law and with 
applicable local stamtes and regulations.^ 

• Ms. Gist has assigned a member of her staff to act as the liaison to the Department and 
WSF and has given her primary responsibility for ensuring that the District’s oversight 
obligations under the follow-on agreement are met. 

• Finally, Ms. Gist is developing an infrastructure within her office for tracking the 
compliance of all non-public schools in the District, including those that participate in the 
voucher program, with local regulations^ requiring them to be accredited by a recognized 
independent accreditation body or to have otherwise demonstrated satisfactory evidence 
of program quality. 

In addition, I have directed the District’s Department of Consumer and Regulatory Affairs 
(“DCRA”) to conduct inspections at all of the schools identified in your report as having 
inapplicable or missing Certificates of Occupancy. As we discussed, these certificates are 
critical, because they show that the schools have met certain construction, health, safety, and fire 
codes.'* Those inspections are underway and should be completed by the middle of next week. 

At that point, we will determine whether those facilities currently meet the regulatory 
requirements applicable to school buildings and can be issued the proper certificates, whether 
any violations that exist can be quickly remedied, or whether a school that is unable or unwilling 
to remedy its status will be required to close or relocate in order to protect the health and safety 
of its students. My office also will work with DCRA to develop an accelerated schedule for 
reviewing the regulatory status of all of the schools receiving funding under the voucher 
program. 



‘ See Consolidated Appropriations Act for Fiscal Year 2004 (H.R. 2673)/P.L. 108-199, and House Rept. 108-401, 
Div. C, Title III. 

^ The MOU stipulates that only schools “operating lawfully in the District" may participate in the program. 

’ See DCMR 2100.2. 

^ DCMR Title 12, 2003 Building Codes Section 1 lOA and DCMR Title 1 1, Section 3203. 



1350 Pennsylvania Avenue NW, Washington, D.C. 20004 



Page 66 



GAO-08-9 District of Coiumbia Opportunity Scholarship Program 






Appendix III: Comments from the Office of 
the Mayor of the District of Coiumbia 



I would like to reiterate that the primary concern of the Fenty Administration is ensuring that the 
instruction received by children who participate in this program meets widely-accepted, 
independent standards for educational programs, that the children learn in facilities that do not 
present threats to their health or safety, and that their parents receive accurate information 
regarding the quality of that instruction. We look forward to working with the Department and 
WSF to ensure that that is the case. 

Thank you, again, for your help in highlighting key areas on which we should focus as we 
developed policies and procedures for effective oversight of this program and as we move 
forward to implement them. 



Sincerely, 






Victor Reinoso 
Deputy Mayor 
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UNITED STATES DEPARTMENT OF EDUCATION 

OFFICE OF INNOVATION AND IMPROVEMENT 



October 23, 2007 



Ms. Cornelia M. Ashby 

Director, Education, Workforce and Income Securities Issues 
Government Accountability Office 
441 G Street, NW 
Washington, DC 20548 

Dear Ms. Ashby: 

Thank you for the opportunity to comment on the draft report entitled, “District of 
Columbia Opportunity Scholarship Program (OSP): Additional Policies and Procedures 
Would Improve Internal Controls, Program Operations, and Goal Attainment.” 

In the report, GAO makes recommendations that the Secretary of Education direct the 
Washington Scholarship Fund (WSF) to improve internal controls, integrate financial 
systems, improve monitoring and site visit procedures, and provide accurate information 
to parents. GAO also makes two additional recommendations that the Secretary and the 
Mayor of the District of Columbia (District) take specific steps to ensure participating 
schools are in compliance with all relevant District education and safety requirements and 
that the Mayor implement procedures to ensure that private schools meet District 
requirements. GAO also recommends that, in planning the implementation of future 
programs for which an evaluation is required, the Secretary take measures to ensure that 
oversight efforts and other program implementation matters are handled in a manner 
consistent with the evaluation design required by Congress. 

The Department’s comments include several general comments on three broad concerns, 
and then comments on specific recommendations. 

GENERAL COMMENTS 

As an initial observation, we feel obligated to note that GAO refused to brief the 
Department on its proposed findings and recommendations prior to sending us the draft 
report. This type of briefing is usually done by GAO at a substantive “exit conference” 
which is held after GAO’s audit work is completed and before the draft report is prepared 
and issued, to provide an opportunity for the parties to discuss the draft findings and 
recommendations and clarify possible ambiguities or misunderstandings, before those 
findings and recommendations are finalized in the draft report. At a July 10 meeting 
GAO staff orally presented what they indicated were preliminary observations and 
indicated that they could not discuss draft findings or any recommendations at that time. 

400 MARYLAND AVE.. S.W., WASHINGTON, DC 20202 
www.ed.gov 
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Following that meeting. Department staff were sufficiently concerned about possible 
misunderstandings by GAO on several key points (including, but not limited to, the 
participation of students from schools in need of improvement, the lawful operation of 
private schools in the District and related matters including teacher quality and 
accreditation, and the program’s evaluation) to request a more substantive exit 
conference. We also submitted a 10-page document to GAO on July 26 with clarifying 
and additional information in a further effort to correct possible misunderstandings. 
Additionally, at the request of a GAO Deputy Assistant General Counsel, on August 6, 
2007, Department representatives participated in a teleconference to address GAO 
questions about the permissibility of paying fees for before and after school programs 
under the OSP. 

Department representatives continued to request that GAO participate in a more 
substantive exit conference meeting with our staff prior to preparing the draft report to 
ensure there were no further questions about the information provided and no further 
misunderstandings to be resolved before a complete and accurate report could be 
presented. Unfortunately, GAO denied the Department’s request for such a meeting and 
the resulting draft report demonstrates that certain misunderstandings continue to exist, 
especially with regard to the information provided to GAO by the Department after the 
meeting held on July 10. 

Additionally, the Department continues to be extremely disturbed that this draft audit 
report was improperly disclosed to various local media outlets, including The Washington 
Post immediately after the issuance of the report. As discussed in General Counsel Kent 
Talbert’s letter to the Comptroller General, dated October 12, 2007, while we do not 
believe that any GAO personnel were involved in the improper disclosure, we are very 
concerned that GAO may not have adequate procedures in place to “prevent improper 
disclosure,” and to follow up, once an improper disclosure is discovered. 

Under section 6.45 of the “Government Auditing Standards: July 2007 Revision,” it is 
imperative to have a secure process for distributing “a draft report with findings for 
review and comment by responsible officials of the audited entity and others” to help “the 
auditors develop a report that is fair, complete, and objective.” When this process is not 
followed and information is disclosed improperly, the fairness, completeness, and 
objectivity of the process are seriously threatened. As a result. General Counsel Kent 
Talbert’s letter requested GAO to investigate the improper disclosure immediately. He 
has received no response to his letter from the Comptroller General. 

Finally, the draft report does not present a complete and balanced picture in a number of 
key areas, and does not accurately reflect what actually occurred in the program during 
the period audited. For example, the report criticizes the extent to which students who 
participate in the program previously attended schools in need of improvement (SINT). 

As previously explained to GAO, the statutory requirement that SINI students receive 
priority under the program was implemented through “scholarship lotteries.” Because the 
lotteries must be conducted in the spring before each upcoming school year, several 
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months before the District reports its SINI designations each August, the priority group 
categories in the lottery are based on SINI designations for the previous year.' In the first 
year of the program, the lottery occurred before there was a meaningful determination of 
SINI, 

Thus, especially for the first two years of the program, it is more accurate to consider the 
designation of the school for the school year in which a student spends his or her first 
year in the OSP-August 2004 SINI designations for the spring 2004 applicants and 
August 2005 designations for the spring 2005 applicants. The distinction is most clear in 
the second year evaluation report. At the time of the initial lotteries, only 5.9 percent of 
spring 2004 applicants were technically from schools designated as SINI for the 2002-03 
school year, but 37.1 percent of the applicants were from schools designated as SINI in 
August 2004 based on those schools’ performance in the 2003-04 school year. 

GAO was asked to assess the implementation of the OSP program, and we believe that 
with regard to a number of key areas, in addition to the one mentioned above concerning 
schools in need of improvement, the draft report presents incomplete and, therefore, less 
than balanced views in a number of important respects, including the following: 

1) The success of the Department and WSF in establishing the OSP in time for the 2004- 
2005 school year. As Congress considered the bill that would authorize the OSP 
program, the Department took steps to identify the internal resources that would be 
needed to effectively implement the program. Once Congress appropriated funding, 
the Department was able to conduct a grant competition and award a grant to WSF in 
an extremely expeditious manner — this whole process took only six weeks, but was 
done in accordance with all appropriate Department policies and procedures. WSF 
very promptly and effectively identified interested private schools and formulated 
agreements with those schools to participate in the program, conducted an 
extraordinary outreach effort to identify families who might be eligible to participate, 
assisted these families in preparing paperwork necessary to document eligibility, 
provided for and conducted a lottery, and awarded scholarships in time for the 
beginning of the school year. 

2) The very positive responses from parents about their children 's experiences in their 
new schools. Studies by both the Department and Georgetown University cite the 
high levels of parental satisfaction with their experiences with the program and with 
their schools of choice. According to the Department’s report, using a scholarship 
significantly increased parental satisfaction with their child’s school in every 
measured area. Seventy-four percent of scholarship parents gave their children’s 
schools an ‘A’ or a ‘B’ grade. In the Georgetown University study, the majority of 
parents reported being very satisfied with their school choice experiences, citing 



‘ For example, SINI students in the cohort 1 (spring 2004) lottery had to be grouped based on SINI 
designations made in August 2003, using performance data from the 2002-03 school year. But 2 months 
after the lottery, significantly more schools were designated for SINI for 2004, and it is this later number 
that provides a more accurate representation of which schools were low performing when students in cohort 
1 were applying to the OSP. 
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changes in their children’s attitudes about learning as the main source of their 
satisfaction. Approximately 90 percent of the parents interviewed reported that their 
children would remain in the program for at least another year. 

3) The increased demand for scholarships in each year of the program ’s existence. 

Most recently, the demand for scholarships rose 5.5 percent for the 2007-2008 school 
year over the previous year. There have been similar increases in demand for the 
program in each of the previous years, and there is now a waiting list of 400 families. 

4) The extent to which private schools participating in the program are legally permitted 
to maintain their traditional independence in such areas as hiring and establishing 
qualifications for teachers. ^ Consistent with the intent and language of the law, the 
program was implemented by WSF in order to encourage maximum participation of 
private schools to ensure sufficient capacity and options for participating families. 
Private schools participating in the program were permitted to maintain their 
traditional and lawful independence in such areas as hiring and establishing 
qualifications for teachers, establishing admissions criteria, and setting standards for 
grading and promotion. We also note that with regard to teacher qualifications, 
District regulation CDCR 5-2100. 2(c) provides: 

(c) Qualifications of staff: training and educational requirements for teaching and 
supervisory staff must be acceptable to the Board, although a District of Columbia 
teaching certificate is not required. 

5 ) GAO’s understanding of WSF explanations for certain decisions. GAO mistakenly 
concluded that “WSF did not adhere to its own procedures for making scholarship 
payments” based on WSF’s decision to use scholarship funds to pay tuition for 
students attending schools that normally receive donor contributions to cover tuition. 
As WSF previously explained to GAO, it believes that this is “a difference of 
interpretation of the statute” since these schools are not “free” but operate on 
contributions from private donors. The OSP provided funding so that eligible families 
could choose to attend these schools and the schools could serve these children when 
they would not have had the financial resources to do so otherwise. 



^ We also note that responsibility for many of the other concerns mentioned throughout the GAO report 
about decisions made during the implementation of the OSP by the Department, the District, and WSF 
were outside of the control of those Implementing the program, e.g., the number of high school slots 
available for scholarship recipients. 

GAO raised a concern about the number of students receiving scholarships during the first year of the 
program that already attended private schools. These students met the definition of “eligible student” in the 
statute, and Congress did not limit participation in the program to eligible students attending public schools. 
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COMMENTS ON SPECIFIC RECOMMENDATIONS 
Internal Controls and Financial System 

The Secretary shall direct WSF, the grantee, to establish and implement detailed policies 
and procedures to improve financial controls over OSP grant funds, establishing 
compensating controls to reduce the risks of fraud in situations where segregation of 
duties is not possible, and continue its efforts to implement an integrated financial 
management system. 

Since the very early implementation of the program, WSF has made significant progress 
in establishing more formal policies and procedures for improved internal controls and in 
integrating its financial management systems to greatly reduce the risk of waste, fraud, 
and abuse. The organization’s annual independent financial A-133 audits since its receipt 
of the federal grant have identified no reportable conditions or material weaknesses. In a 
separate letter, WSF’s auditor recommended, and the organization subsequently 
developed, a financial manual consolidating its policies and procedures. This manual 
currently is being updated to include further details recommended by GAO. 

WSF has explained to GAO that its new billing system is fully integrated with its student 
information database and has replaced the manual procedures to which GAO refers in its 
draft report. A fully improved financial system is scheduled to be in place by 
November 1 and will be updated with all data for the 2007-08 school year. Department 
staff will continue to work with WSF to ensure these policies and procedures are 
appropriately implemented and updated, as necessary. 

School Monitoring and Site Visits 

'The Secretary shall direct WSF to develop and implement procedures for conducting site 
visits. 

Regarding GAO’s recommendations for site visits by WSF, the organization keeps 
records of all visits. WSF has explained to GAO that it extensively documents problems 
or issues that are identified during site visits, and takes timely corrective action. As a 
control to check the accuracy of assurances and directory data, WSF is developing 
specific site visit questions and related procedures. 

Information to Parents 

The Secretary shall direct WSF to develop procedures to ensure that accurate 
information is provided to parents before a school is chosen. 

On a related GAO recommendation regarding school information provided to parents, 
WSF has consulted with the private schools on an annual basis to collect and update this 
information. During the first year of the program, WSF asked each school to complete a 
survey and in subsequent years asked each school to edit a printout of the previous year’s 
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information and provide updated data. As referenced above, WSF will include data 
verification questions when it conducts site visits as a control to determine the accuracy 
of information that schools have reported for the directory for parents. 

Collaboration with Mayor of the District of Columbia 

The Secretary shall collaborate with the Mayor of the District of Columbia to ensure 
participating schools are in compliance with all relevant District of Columbia education 
and safety requirements. 

GAO also recommends that the Secretary collaborate with the Mayor of the District of 
Columbia to ensure that participating schools are in compliance with all relevant District 
of Columbia regulations, including education and safety requirements. The Department’s 
July 26 written submission provided detailed information documenting that, given the 
guidance set forth by Congress in the statute and accompanying legislative history 
regarding private school participation in the program, the approach taken by the 
Department and the District in the Memorandum of Understanding (MOU) - which 
requires participating schools to provide assurances that they are operating lawfully 
within the District - is lawful and reasonable and fulfills the joint oversight 
responsibilities set forth in the federal statute. This approach of using assurances is 
commonly utilized in a number of Department programs and is often required by law in 
programs administered by the Department. The approach was designed to reduce burden 
and maximize participation by eligible private schools in the District while ensuring 
compliance with all applicable legal requirements. Additionally, we note that WSF does 
have an established process to review practices of private schools participating in the 
program in order to address instances when concerns about participating schools were 
identified. 

WSF’s implementation of the requirements identified in the federal statute and MOU 
through its guidance and forms (including its Letter of Agreement, Key Data Form, 
consultation with schools, and ongoing consultation with the Department about the 
process of school consultations and complex questions raised during them) were an 
appropriate and effective exercise of grantee authority. The Department did not have any 
reason to assume that assurances would not fully reflect the school’s legally operating 
status in the District. Given the results of this study, however, the Department is taking 
steps to ensure that the participating schools are in compliance with all District 
regulations. 

Department staff and the new State Superintendent of Schools (now the District’s 
responsible official for the Opportunity Scholarship Program under the MOU) already 
have met to discuss and clarify appropriate roles for the OSP. An important area of 
cooperation will be ensuring that all schools meet the requirements of the District for 
operating lawfully. We understand that the Mayor’s office has directed the Department 
of Regulatory and Consumer Affairs to conduct inspections, on an expeditious basis, of 
the schools that GAO identified as not having occupancy permits. Also, the State 
Superintendent of Schools office is assuming responsibility for ensuring that there is 
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appropriate documentation for the accreditation of private schools, including those 
participating in the OSP. The Department and the State Superintendent of Schools office 
will revise the current MOU to reflect these specific activities and any other 
responsibilities agreed upon by the two agencies. 

Program Implementation and Evaluation Design 

Finally, we address GAO’s findings and recommendations on evaluation. GAO asserts 
that program implementation and other factors have limited the usefulness of and ability 
to generalize its evaluation findings, and that in the future, the Department should ensure 
that program implementation is more consistent with the design of a statutorily mandated 
evaluation. 

While we agree that some external factors did have an effect on carrying out the 
evaluation as planned, GAO appears to have misunderstood some aspects of the 
evaluation, leading the agency to underestimate the evaluation’s utility. Below, we have 
addressed GAO’s concerns by correcting misimpressions about the evaluation design in 
general and its priorities. 

1) The inability to compare OSP participants to all students in DCPS schools does not 
undermine the more important analysis of program impacts. GAO rightly points out 
that the change in the testing program by the DC Public Schools (DCPS) prevents the 
evaluation from comparing the academic achievement of scholarship recipients to the 
achievement of students in DCPS in general. However, that was only one of the 
analyses called for in the legislation, and it does not provide a reliable indicator of 
program effects because, as the evaluation’s first year report described, DCPS 
students who did not apply to the OSP are different from those who applied and are 
participating, making any simple comparison between outcomes from the two groups 
biased and unscientific. The main purpose of the evaluation was to estimate the 
effectiveness or impacts of the program, comparing the achievement of OSP 
applicants who did and did not receive scholarships through the lotteries, and this 
purpose is being fulfilled. 

2 ) The inability to include some early scholarship recipients in the evaluation is not a 
major problem as GAO implies. Very early applicants to a program are often 
different (e.g., more motivated or more needy) than those who apply in subsequent 
years. It is tme that the impact evaluation could not include the first year OSP 
applicants for grades K-5 because they were all given scholarships and therefore no 
control groups were formed for them through lotteries. However, the set of second 
year (cohort 2) applicants — all of whom are included in the evaluation — is likely to 
provide a more reliable estimate of impacts for the types of students who would be 
participating when the program is in a “steady state” of operations. For that reason, 
the eviuation team considered restricting the impact analysis to cohort 2. We 
included the cohort 1 randomly assigned middle and high school applicants so that 
there would be sufficient samples of students to allow impacts to be estimated for 
subgroups of students (e.g., by grade band), but we statistically allow for differential 
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effects across the cohorts. Even without the K-5 first year applicants, the number of 
students included in the impact evaluation is actually the largest used in any rigorous 
study of private school scholarships. 

3) "Crossover” is a natural part of any program and randomized control trial, not a 
flaw in them. GAO describes the fact that some control group students (those who 
applied but did not receive a scholarship through the lotteries) attended private 
schools as hindering the analysis and interpretation of program impacts. In fact, the 
control group’s behavior is supposed to reflect what would have happened in the 
absence of the program; students’ ability to obtain access to a scholarship other than 
the OSP or to find some other way to attend a private school without the OSP is an 
important piece of information in determining the impact of the offer of this 
scholarship. The evaluation then uses commonly accepted statistical approaches to 
“net out” the crossovers in its analysis of the relationship between attending a private 
school and student outcomes, which is separate from the analysis of program impacts. 

GAO also criticizes the evaluation for failing to distinguish scholarship recipients 
who did not use their scholarships because they could not find a school placement 
(“forced decliners”) from non-users who declined to use their scholarship for other 
reasons. Since GAO officials reviewed the evaluation’s analysis plan, they know that 
the study intends to address this issue through a variety of non-experimental 
approaches. However, the first year there were so few of these forced decliners that 
the issue did not warrant additional analyses; these analyses are being incorporated 
into the next report describing impacts after two years. 

4 ) The concentration of scholarship recipients ( including those in the impact analysis) 
in a subset of private schools does not interfere with accurate estimation of program 
impacts. It is neither surprising nor unique to the DC OSP that participating students 
are not evenly distributed among the participating private schools. As a policy 
intervention, school scholarships provide resources to parents who subsequently 
select private schools for their children. Those participating private schools will 
inevitably vary in the number of spaces available to scholarships students and in their 
relative attractiveness to parents. That is, choices made by both private schools and 
parents are a part of the OSP “treatment.” There is no “confounding” of program and 
school effects, as GAO states, because the school effects and the program effects are 
one and the same; any statistical techniques to control for school-level characteristics 
in evaluating school choice programs would improperly take away most if not all of 
the true treatment effect. In technical terms, the clustering of OSP scholarship users 
within private schools only reduces the efficiency of the impact estimates, and only 
by 2 percent as described in the Year 1 impact report, but the substantial number of 
students in the impact sample helps to offset that slight reduction in precision. 

5) The other supports the program operator provided were not organized or used in 
ways that lessen the usefulness of the evaluation findings. If the program operator 
(WSF) systematically provided supplemental services (beyond the offer of a 
scholarship), then the evaluation would be estimating the impacts of that combined 
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treatment and would have a difficult time disentangling the independent effect of the 
scholarship. However, as we understand it, WSF makes additional services available 
to some OSP recipients by referring them to existing community-based organizations, 
but not all recipients take up the offers. For example, WSF reported that fewer than 
20 of about 80 high school first-year scholarship users went through the process to be 
assigned to mentors. A foundation provided tutoring services at two or three 
Archdiocese schools, and some OSP students took advantage of that. But it is 
important to note that members of the control group also had access to and availed 
themselves of these kinds of programs and services offered through their public 
schools or generally in their community. In fact, random assignment should have 
ensured that students in both the treatment and control groups were equally motivated 
to take advantage of the supplemental services available to them. For the upcoming 
second impact report, the evaluation team intends to estimate the impact of the 
program on participation in other services, but the availability of these services to 
both groups does not undermine the validity of the evaluation findings. 

Thank you for the opportunity to provide a written response to the draft report. We 
strongly encourage GAO to consider the information provided in our July 26 submission 
and in this response when preparing the final report on the District of Columbia 
Opportunity Scholarship Program. Additionally, we think it would be very helpful if 
GAO provided Department officials the substantive meeting that we requested earlier. 

We feel that such a meeting will result in a thorough and balanced audit report that does 
not include apparent misunderstandings that we have noted above. In any event, as 
indicated, we will use the information in this report to continue to improve the 
Department’s oversight of the program and its coordination with the D.C. State 
Superintendent’s Office. 



Sincerely, 






MofgfiftS. Brown 
A^istaiit Deputy Secretary 
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October 23, 2007 



Cornelia M. Ashby 
Director, Education, Workforce, 
and Income Security Issues 
Government Accountability Office 
441 G Street, N.W. 

Washington, D.C. 20548 

Dear Ms. Ashby: 

Thank you for permitting the Washington Scholarship Fund (“WSF”) an opportunity to comment 
on the draft Government Accountability Office (the “GAO”) report entitled, “District of Columbia 
Opportunity Scholarship Program: Additional Policies and Procedures Would Improve Internal 
Controls, Program Operations, and Goal Attainment” (the “Draft Report”) - a copy of which was 
forwarded to WSF, under strict confidentiality requirements, on October 9, 2007. WSF appreciates the 
thoroughness of the Draft Report - and the GAO’s express acknowledgment that the Draft Report 
addresses what essentially was the “start-up” phase of the D.C. Opportunity Scholarship Program (the 
OSP). That is, the Draft Report provides, by apparent consensus, a “snapshot” of a program that 
experienced initial challenges, met them, and then thrived. 

We are concerned, however, that the Draft Report does not present an accurate picture of the 
OSP’s great successes to date, and - from a strictly objective perspective - contains several factual 
errors and incorrect assumptions and conclusions [based at least in part on what might be an erroneous 
interpretation of the OSP’s federal authorizing statute (the “OSP Statute”)]. Further, the Draft Report 
focuses primarily on matters that, in the vast majority of instances, frankly bear little on the financial 
viability and effectiveness of the OSP. In fact, the Draft Report’s purported findings, emphases, and 
other points often have little to do with the matters proposed for the GAO audit by those requesting the 
audit in the first instance (or, for that matter, with the key objectives of the OSP Statute). 

As has been widely reported - including by the U.S. Department of Education (the “DOE”) in its 
report released this past June - the extremely low-income families participating in the OSP over the past 
three-plus years have been overwhelmingly satisfied with the program and, more crucially, with their 
chosen schools. This evidence of parental satisfaction with the OSP is corroborated by a report released 
in May of this year by Georgetown University, which also found that OSP students are more engaged in 
their school work and have increased self-esteem, and that OSP parents are more involved with their 
children and their chosen schools. In fact, in its first three-plus years of operation, the OSP has fulfilled 



1100 17* Screet, NW, Suite 330 ♦ Washington, DC 20036 ♦ 202.222.0535 (p) ♦ 202.222.0543 (f) 
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all of the key objectives of both the federal authorizing statute' and the Memorandum of Understanding 
(the “MOU”) between the DOE and the District of Columbia (the terms of which largely do or should 
govern the school oversight matters on which the GAO so closely focuses in the Draft Report). 

Ultimately - with reference to what we understood to be the very rationale for the GAO’s audit - 
we must emphasize that the Draft Report cites absolutely no evidence that federal OSP funds have been 
spent for anything other than genuine educational purposes at any time during the OSP’s three-plus 
years of operation, or that OSP families have not received the educational services they sought throuvb 
participation in the OSP. More directly - again, given the GAO’s apparent focus - the GAO preset 
evidence that the financial or other instability of any school participating in the program has deprivt 
any OSP family of the education they sought for their children through an exercise of meaningful 
educational choice. (To the contrary, and again, OSP families overwhelmingly are satisfied with thei, 
schools and their children’s progress and growth in their schools of choice.) 

WSF acknowledges that some of the GAO’s observations and recommendations in the Draft 
Report (and even some of its findings) - particularly those relating to the implementation of the OSP 
during the program’s challenging initial period of operation - are valid and useful. In fact, as the GAO 
is aware (and again, as the GAO has acknowledged), WSF already has taken many of the steps 
recommended by the GAO toward more efficient operation of the OSP. On this point, again worthy of 
great emphasis, and again as acknowledged by the GAO itself: Even before the GAO initiated its year- 
long audit in September of 2006, WSF already had implemented many of the sorts of efficient and 
refined systems required to meet most of the GAO’s recommendations. 

Certainly, WSF will continue to improve the OSP in our steadfast effort to make the program a 
more than viable option for District families seeking quality educations for their children. Naturally, 
toward this end, WSF looks forward to addressing and implementing the GAO’s remaining findings and 
recommendations in close cooperation with both the DOE and the District of Columbia Government — 
both of which have been, and have committed to remaining, close and supportive partners of WSF in our 
continued successful implementation of the OSP. 

We provide below a more detailed response to the GAO’s Draft Report. In our response, we first 
review the questions posed to the GAO by Senators Kennedy and Durbin and Delegate Holmes Norton 
in their initial audit request to the GAO (Section I). We then provide our responses to the specific 
purported factual findings set forth by the GAO in the Draft Report (Section II). Finally, we set forth 
our response to the recommendations made by the GAO in the Draft Report (Section III).' 

' More specifically, the OSP has .served the District’s lowest-income students; it has prioritized students coming from failing 
public schools as determined by federal No Child Left Behind Act criteria; and it has supported the rigorous evaluation of the 
OSP mandated by the federal authorizing statute. 

^ For example, among the primary points on which WSF disagrees with the GAO is the GAO’s assumption that WSF should 
function as a “Super School Board” that subjectively determines the educational value of each .school participating in the 
OSP. Such a role for WSF is permitted neither by the OSP Statute, nor by D.C. law, nor by any other applicable statute or 
regulation - and in any event, under any circumstances, would be wholly inappropriate. 

’ In the interests of economy and relevance, WSF will not attempt to respond to each and every factual, logical, or other error 
in the Draft Report. Rather, we focus in our response on those Draft Report inaccuracies and weaknesses that appear most 
likely to create a misleading impression about the overall functioning and success of the OSP, and that most require 
correction or attention in the public interest. 
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I. Questions Posed to the GAO, and Scone of the Resulting Audit/Investigation 

As stated in the Draft Report, the GAO was requested to answer/address three specific questions/matters 

(Draft Report, p. 2): 

To assess the implementation of the program, including the (I) accountability 
mechanisms in place governing the use of funds, (2) results of the grantee 's efforts to 
meet the program 's recruiting and eligibility requirements and inform parents about 
their choices, and (3) extent to which the evaluation reflects statutory requirements and 
the implementation of the program supported the detection of useful and generalizable 
findings. 

The GAO in its Draft Report has not addressed the results of WSF’s efforts - and successes - in meeting 

the OSP Statute’s priorities: 

1. Give priority to students coming from schools identified for improvement 

In year one of the program, 29% of OSP scholarship recipients had been identified as having come 
from schools identified as in need of improvement (“SINI” schools) in 2003 or 2004. Only 4% were 
identified at the time of the initial lottery because the 2004 list of SINI schools was published after 
the independent federal evaluators, under the DOE’s Institute of Educational Sciences’ (“lES”) (the 
“lES Evaluators”), held the first lottery on June 17, 2004. (The lES Evaluators described this 
situation in their first-year report on the OSP). 

In the current 2007-2008 school year, 83% of students participating in the OSP would be in schools 
that have failed to meet adequate yearly progress if it was not for their OSP scholarships. 

In the initial OSP lottery, the lES Evaluators - who designed and administered the lottery - gave 
priority to students in SINI schools. In year one of the OSP: 

■ 100% of eligible OSP applicants from a school needing improvement received a scholarship; 

■ 2 out of 3 eligible students from public schools in grades 6 to 8 received a scholarship; and 

■ 3 out of 10 eligible students from public schools in grades 9-12 received a scholarship. 

2. Target resources to families that lack the financial resources to take advantage of available 
educational options 

■ Nearly 20% of eligible District smdents have applied for OSP scholarships in the program’s first 
four years of operation. 

3. Provide students with the widest range of educational options 

■ The GAO itself reports that 78 of the 86 non-public schools in the District (80%) participate in 
the OSP. 
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II. WSF’s Responses to the GAO’s Purported Factual Findings 

In this section, WSF will state the GAO’s purported factual findings (numbered and in bold italics), and 
then, seriatim, will set forth our responses to those purported findings. 

1. GAO Purported Finding; WSF’s internal controls policies and procedures and systems did not 
provide adequate accountability over the use of grant funds 

WSF acknowledges that during the initial start-up of the OSP, WSF’s internal control policies and 
procedures, systems, and internal control activities were not optimal, including: 

■ Several procedures and electronic systems that relied on manual data transfers, increasing the 
risk of human error; and 

■ Policies and procedures which were not documented in detail. 

As the GAO notes (Draft Report, pp. 12 and 14), “Due to the need to quickly implement the 
program, WSF had little time to develop internal control policies and procedures, systems, and 
internal control activities,” and “had to rely on the knowledge of key staff to perform daily 
operations.” The demands placed on WSF and our staff in attempting to get the OSP up and running 

- within five months of the date WSF was appointed the administrator of the OSP and the first day 
of the then-upcoming (2004-2005) - were enormous. As the OSP includes many characteristics 
unique and innovative to scholarship programs, WSF’s policies, procedures and systems evolved as 
the program matured. This learning curve made documenting policies and procedures as they 
evolved nearly impossible. In addition, the limited operating funds provided under the OSP Statute 

- which provide only about 16% of the actual cost of operating the OSP and serving our students and 
families - did not afford WSF the staff resources necessary to both create and implement the 
program, and to document our efforts simultaneously during the 14-month launch period. Indeed, 
WSF has been required to raise substantial private funding in order effectively to operate the OSP, 

WSF is appreciative of the GAO’s recognition that WSF has already made major improvements in 
these areas, having made significant financial investments in systems and infrastmeture 
improvements in advance of the request to GAO for the program review. To date, WSF has nearly 
completed the full electronic and real-time integration of its data (already operational) and financial 
systems (launching November 1 , 2007). The appropriate detailed policies and procedures will be 
finalized once tandem financial systems have been in operation for two months (January 2008). 

Further, WSF welcomes the GAO’s suggestions for additional operational improvements as set forth 
on pages 1 7- 1 8 of the Draft Report concerning revisions to its Internal Systems Review, and will 
adopt those suggestions. 

2. GAO Purported Finding: As a result of the manual and disparate policies, procedures and 
systems, some payments were not made in accordance with the act, while others raise questions 
about compliance 

While WSF agrees that the deficiencies in internal controls and accountability mechanisms increased 
the risk of errors, WSF firmly represents that no federal monies were spent irresponsibly or 
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improperly used for purposes other than OSP scholarships. The GAO specifies three areas where it 
believes that the lack of internal control mechanisms resulted in errors in the use of federal 
appropriations (pp. 12 & 21): 

(a) WSF’s payment of scholarships for schools that are “scholarship-only” 

Payment of scholarships for schools which GAO identifies as “scholarship-only” was a 
result of considered choices, not a result of any lack of internal control mechanisms. 

WSF believes that this choice is in complete accord with the OSP Statute. 

Three participating OSP schools (Washington Middle School for Girls, San Miguel, and 
Washington Jesuit Academy) are “scholarship-only” schools - that is, the educational costs for 
all of the children who attend those schools arc paid for by scholarships funded by third-party 
donors. GAO contends that allowing OSP scholarships to be used for these programs violates 
Section 307(a) of the Act, D.C. Code § 38- 185 1.06(a)(1), which provides in part that: 

Each grantee shall ensure that the amount of any tuition or fees charged by a school 
participating in the grantee ’s program under this chapter to an eligible student 
participating in the program does not exceed the amount of tuition or fees that the 
school customarily charges to students who do not participate in the program. 

WSF respectfully submits that the GAO’s reading of this provision is legally incorrect. GAO’s 
premise is that if a student receives a scholarship to a school and thus does not personally pay 
tuition, then allowing any use of OSP funds for that school would not be permissible because 
non-scholarship students “pay no tuition.” But this argument cannot be what Congress intended, 
since virtually all schools that participate in the OSP provide scholarships to some students. 
GAO’s reading of Section 307(a)(1) would mean that students who receive scholarships at other 
schools are viewed as “paying tuition” for purposes of Section 307(a)(1). There is no logical or 
legal basis for applying a different standard to students attending the three “all- 
scholarship” schools receiving non-OSP funding than the standard applied to students 
receiving non-OSP assistance attending any other participating school. 

The purpose of Section 307(a)(1) is to prevent a school from “overcharging” OSP students - a 
concern that is absent in the “all-scholarship” situation. In the cases of the above-cited schools, 
the simple fact is that tuition is not normally paid by the families, but by other sources similar to 
OSP scholarships. In fact, the above-cited schools list tuition amounts for donors and on their 
websites and in other materials readily available to the public. As with most other participating 
OSP schools, those published tuitions are below the actual costs of serving the students in those 
schools. 

While the Draft Report does not specify the reasoning behind the GAO’s objection to 
allowing OSP scholarships at the “all-scholarship” schools, the draft report refers to these 
schools as “free.” This reflects a misunderstanding of the way these schools operate. 

They are not, in fact, “free” but require money to operate which they collect from public 
sources and private donors. If OSP students did not have OSP scholarships to fund their 
educations at these schools, those student could not attend unless other donors stepped 
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up, diverting funds from a non-OSP student. In other words, OSP scholarships provide 
the resources for these schools to educate additional children (i.e., the OSP students). 

WSF does not believe that it would discharge - and indeed would defy - its statutory 
responsibility if it disqualified these three schools from participation in the OSP simply 
because all of these schools’ students receive scholarships. 

In addition, the GAO states that by listing the published tuitions for these three schools in 
the WSF School Directory, parents are misled to believe that they could not attend/afford 
those schools if they did not have an OSP scholarship. As OSP families arc provided a 
copy of the WSF School Directory when they have completed their initial application for 
an OSP scholarship, however, this could not logically affect a family’s decision to apply. 

(b) WSF allowing scholarship funds to be used for after-school programs without demanding 
proof of “academic support activities” 

Again, WSF notes that this represents a considered choice by WSF in its implementation of the 
OSP — not any lack of internal control mechanisms. WSF believes that this choice was and 
remains fully in accordance with the OSP Statute.’* School fees are defined under the OSP 
Statute as “fees charged by the school that the student is emolled in,” and “are charged at the 
same rate as non-OSP students” - in contrast to the GAO’s own definition, which docs not 
comport with the governing federal statute. 

WSF permits the use of OSP scholarship funds for before- and after-school programs that are 
offered to other, non-OSP students at participating schools. The GAO states (Draft Report, p. 

1 2) that a “key” element of an allowable fee under the OSP Statute and WSF policy is that the 
fee is “tied to the student’s academic program,” The GAO further states (Draft Report, p. 21) 
that unless there is proof that these programs “included any academic support activities,” then 
“before-and-aftcr care would not meet the requirement that a fee must contribute to a student’s 
academic success.” 

WSF respectfully submits that the GAO appears to have misunderstood and, therefore, misstated 
WSF’s policy for school fees (as well as the OSP Statute’s mandate). School fees are defined as 
those that support the academic success of the student. WSF vigorously disagrees with the 
GAO’s conclusion that there must be formal “academic support activities” in order for before- 
and after-school programs to contribute to students’ success. Many studies have shown that 
before-school and after-school programs play an important role in student academic success by 
providing a supportive and supervised environment. For example: 



■' Thus, WSF does not believe that these two legal issues are within the scope of the inquiry requested by the Members of 
Congress into “accountability mechanisms.” 
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Research Findings: Studies on the Effects of After-School Programs on Students’ Academic 
Success and Sociai Integration 





Findings 


Study 


Dr. Joseph Mahoney and 
Dr. Heather Lord, 

Yale University 


“Children characterized by the After- 
School Program (ASP) care arrangement 
showed significantly higher reading 
achievement at the end of the school 
year compared with children in all other 
patterns of after-school care [e.g. 
relatives] and were rated by teachers as 
holding greater expectancies of success 
compared with children in the other 
adult/non-adult care pattern.” 


An Ecological Analysis of 
After-School Program 
Participation and the 
Development of Academic 
Performance and Motivational 
Attributes for Disadvantaged 
Children. 


Erica Carryl, 

New York University 


“The finding supports a conclusion that 
among the typical after-school care 
arrangements poor children experience, 
ASPs appear unique in their ability to 
promote academic-related success.” 


Published in 2005 by the 
Society for Research in 
Development, Inc. 



(c) Certain alleged paperwork errors in the sample of scholarship files reviewed by the GAO 
caused OSP fiinds to be snent that were not in accordance with the Act 



WSF believes that the deficiencies noted are minor in nature, and do not indicate substantive 
problems with the disbursement of federal funds. 

The GAO reviewed a sample of 50 student files for the 2005-2006 school year, and asserts (Draft 
Report, pp. 19-20) that it found certain deficiencies in the files: (i) Missing signatures or dates 
on fee approval forms; (ii) missing documents, such as school placement letters; and (iii) missing 
student identifier numbers. WSF has not been provided with the specifics of these alleged 
deficiencies and thus cannot respond in detail. WSF notes, however, that most of the alleged 
deficiencies arc non-substantive (e.g., missing dates, missing student identifier numbers) and do 
not reflect erroneous distribution of federal funds. 

The GAO does indicate that it found one instance of payment of a fee that was not listed on the 
payment detail report, and one instance of a payment made on the basis of a payment detail 
report that had not been certified by a school official. Again, WSF does not know the details of 
these two situations, or the dollar amounts involved - but regardless, and again, these situations 
under any objective analysis must be considered only minor clerical oversights. 

WSF also would submit that the incidence of error is inflated by the GAO’s counting a missing 
signature on one fee form as 20 errors if the fee was charged to 20 students’ scholarships. 
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J. GAO Purported Finding; While OSP students generally met the program ’s eligibility 
requirements, WSF (a) had limited success in meeting recruiting priorities and (h) did not 
provide parents compiete information 

(a) Recruiting priorities 

WSF shares the concerns about outreach to District students in “schools in need of 
improvement,” and certainly has sought assiduously throughout the first three-plus years of OSP 
implementation to prioritize students coming from failing schools. But given the newness of the 
No Child Left Behind Act at the time of the OSP’s initiation in 2004, and the timing of the 
program’s implementation with respect to testing requirements in the District, no school could 
have been identified as “in need of improvement” by the time the lES Evaluators needed to 
perform the first OSP lottery (it takes two years of a school failing to meet “adequate yearly 
progress” for the school to be identified as “in need of improvemenf’). 

The lES Evaluators’ report on the OSP for the first year of the OSP identifies this challenge, and 
provides data showing that 29% of OSP applicants for the 2004-2005 school year were enrolled 
the year before in SfNI schools. 

With respect to the analysis in the Draft Report, WSF notes that it cannot be determined from the 
data set forth whether smdents from SINI schools are in fact under-represented among OSP 
students. As noted by the GAO, the great majority of OSP students are at the elementary school 
level, and the percentage of D.C. elementary school students who arc ctu-ollcd in SINI schools is 
much lower than the percentage of D.C. middle school and high school students who are enrolled 
in SINI schools (Draft Report p. 26). Yet when the GAO compares the percentage of OSP 
students from SINI schools to the overall percentage of students from SINI schools, it 
erroneously uses as its benchmark all D.C. students, rather than performing a grade-by-grade 
comparison. This “apples versus oranges” methodology skews the results toward finding an 
under-representation of SINI students in the OSP, when a more appropriate grade-by-grade 
comparison might show very different results. 

WSF also submits that the OSP Statute sets forth no specific smdent reemiting requirements. In 
fact, the only provision in the OSP Statute relating to such matters is contained in Section 
305(b)(1)(G), which requires the grantee [WSF] to address how it will “seek out private 
elementary schools and secondary schools in the District of Columbia.” 

(b) Alleged failure of WSF to perform its financial oversight role 

WSF’s limited oversight role is defined by Section 305(b)(1)(H) of the OSP Statute, which 
requires WSF to ensure that “participating schools arc financially responsible and will use the 
funds received under [the OSP Statute] effectively.” Based on an unsubstantiated and vague 
assertion that “certain documents” were missing from files reviewed by GAO staffers, the GAO 
concludes that WSF “did not fulfill its oversight role of ensuring that participating schools are 
financially stable” (Draft Report, p. 32).’ 



^ It is not clear as to which part of the Congressional audit request the GAO’s discussion of this issue is responsive. 
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WSF first must note that the GAO’s unfounded conclusion rests on a misinterpretation of the 
OSP Statute. The OSP Statute, as cited above, requires WSF to determine the “financial 
responsibility” of schools, not to “ensure” their “financial stability” (which obviously would be 
impossible for WSF to do). 

In any event, WSF emphatically disagrees with any conclusion that it has not performed its 
statutorily- prescribed oversight function. WSF requires the submission of financial information 
from every school; based on those documents and certain statistical “triggers” (such as high rates 
of students choosing to leave a school), WSF undertakes more thorough financial reviews of 
individual schools in cases where it appears that financial problems might exist. As a result of 
such financial reviews, WSF has disqualified one school from further participation in the 
program, and has placed two other schools on a “pay-as-you-go” basis, whereby these two 
schools receive federal funds only after they have provided educational services. As a result of 
these careful financial reviews by WSF, not a single dollar of federal funds has gone to a 
school unable to provide the educational services for which federal scholarship funds were 
spent. 

In addition, despite what the GAO seems to suggest, WSF is unclear as to the ability of a 
school’s clean audit to ensure that the school will be an ongoing enterprise beyond a given 
school year. A financial audit indicates only that an organization has represented its financial 
statements fairly and accurately, and has used industry approved standards; a clean audit by no 
means “ensures financial stability.” 

4. GAO Purported Finding: Alleged failure of WSF to ensure that schools have “certificates of 
occupancy” and have supplied information to the District about their instructional programs 

The GAO asserts in the Draft Report (p. 33) that WSF shares in the responsibility for ensuring that 
private schools are “operating lawfully” in the District of Columbia. WSF disagrees with this 
conclusion. WSF does not have the authority to enforce District of Columbia law; nor, for that 
matter, does WSF have the resources or the expertise to duplicate the functions of the DOE and/or 
the District Government. WSF does, of course, look forward to continuing the very productive 
relationships it has forged with the DOE and the District, and to coordinating on better ways to 
communicate regarding the status of District non-public schools participating in the OSP. 

Further, WSF is concerned by the GAO’s assertion that the GAO’s inability to locate two schools’ 
certificates of occupancy is an indication that certain schools researched by the GAO may not be 
operating legally (Draft Report, pp.8 and 33-34). 

Better perspective suggests that this likely may be a matter of deficiencies in the recordkeeping 
concerning certificates of occupancy and annual reports required by the D.C. compulsory school 
attendance regulations. Tellingly, there is no indication in the Draft Report that the GAO checked 
directly with the schools to determine whether they had been issued certificates of occupancy. In 
fact, The Washington Post noted on Sunday, October 21, 2007, that Pcpco and Verizon will not start 
power or phone services for a school until the D.C. Department of Consumer and Regulatory Affairs 
has issued a certificate of occupancy. See David Nicholson, “D.C.’s Learning Curve,” Washington 
Post, page B4 (October 2 1 , 2007). 



Page 85 



GAO-08-9 District of Coiumbia Opportunity Scholarship Program 






Appendix V: Comments from the Washington 
Scholarship Fund 



Cornelia M. Ashby 

October 23, 2007 

Page 10 

Indeed - as concrete support of the notion that many of the issues broached by the GAO have more 
to do with recordkeeping than with legality - WSF is pleased to report that we have located a 
certificate of occupancy for one of the schools (Academy De La Recta Porta, “ADLRP”) referenced 
in the Draft Report. This certificate clearly indicates that the space used by ADLRP is approved for 
educational use. (The referenced certificate of occupancy is attached hereto as an Addendum.) It is 
apparent that the GAO was unable to locate ADLRP’s certificate of occupancy because the 
certificate was filed under the name of the church that runs the school (New Dimensions Kingdom 
Ministries) rather than under the name of the school itself. WSF located the ADLRP certificate of 
occupancy simply by telephoning ADLRP’s principal, who then forwarded the certificate to WSF 
via facsimile within ten minutes of the phone call, 

5. GAO Purported Finding: Allegation that WSF did not provide parents with “complete 
information” about participating schools 

The GAO asserts that WSF provided parents with incomplete and incorrect information about 
participating schools (Draft Report, pp. 35-36). WSF concurs that some typographical errors existed 
in the WSF School Directory, and will engage in additional proofing upon the next publication of the 
directory. WSF does not, however, regard this directory as the sole or even primary tool used by 
parents in reviewing their educational options. In fact, WSF strongly encourages families to visit 
any school to which they intend to apply, and to talk to other families about their experiences in 
those schools. 

WSF understands that the GAO suggests that participating schools are obligated to publish 
information concerning the academic achievements of their students and that WSF should 
disseminate this information to parents. The OSP Stamte, under Section 310(c), appropriately 
assigns to participating schools the responsibility of reporting to individual parents about the 
performance of students and school safety. 

As the GAO acknowledges, WSF provides parents with information on enrolled students’ academic 
achievement, as measured by a comparison with the aggregate academic achievement of other 
smdents at the student’s school in the same grade or level on an annual basis. Further, WSF follows 
up with each participating school at the end of each school year and receives certification that each 
school provided the required information. 

6. GAO Purported Finding: Evaluation contractors adopted a strong methodology, but program 
implementation and other factors limited the usefulness and generalizability of findings 

For the most part, WSF will not respond to the GAO’s findings in this section of the Draft Report, 
believing that these statistical issues are best left to the DOE and the lES Evaluators. 

WSF is compelled, however, to comment on the Draft Report’s possible implication that WSF 
somehow acted improperly by implementing decisions that were “made to advance program goals,” 
because these steps made statistical assessment of the program’s results more complex (Draft 
Report, p. 44). WSF was instmeted directly by the lES Evaluators in any matters that might affect 
the evaluation, and in fact the lES Evaluators themselves stmetured and implemented the lotteries 
through which the study has been populated. 
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Regrettably, the GAO wholly omits from its Draft Report any discussion of these indicators and 
other evidence clearly showing the success of the program. 

lII. WSF’s Responses to the GAP’S “Recommendations for Executive Action” 

1. Establish and implement policies and procedures to improve financial controls over OSP grant 
funds, including specific requirements for the process of approving scholarship payments and 
documentation of the process 

As acknowledged in the Draft Report, and as discussed in this response, WSF is well underway in 
implementing many of the recommended improvements to the program systems identified by the 
GAO. WSF will complete its policies and procedures manual once this implementation is 
completed. 

WSF has received a clean opinion in all three A-133 audits performed since the OSP was 
implemented, and expects once again to receive a clean opinion in its 2006-2007 audit, currently 
underway. 

2. Establish compensating controls, such as supervisory review, to reduce the risk of fraud in 
situations where segregation of duties is not possible due to the size limitations of OSP’s staff so 
that no one employee can authorize, process, review, and have access to the funds relating to OSP 

Again, as acknowledged in the Draft Report, and as discussed in this response, WSF is well 
underway in implementing the improvements recommended by the GAO, and compensating controls 
and access to financial management systems. 

3. Continue its efforts to implement an integrated financial management system to facilitate 
processing and recording of scholarship payments and overall financial reporting 

As acknowledged in the Draft Report, and as discussed in this response, WSF is well underway in 
implementing many of the recommended improvements to the program systems identified by the 
GAO. WSF will finalize its policies and procedures manual once this implementation is completed. 

As noted above, WSF has received a clean opinion in all three A-133 audits performed since the 
OSP was implemented, and expects once again to receive a clean opinion in its 2006-2007 audit, 
currently underway. 

4. Develop and implement procedures for conducting site visits, including that site visit reports be 
prepared to contain information on the overall financial stability of the school 

WSF does not have the resources to perform a financial analysis “on the overall financial stability” 
of every school participating in the OSP, nor does WSF have the resources to visit every 
participating OSP school every year. 

WSF does look forward to continuing our very close and productive relationships with the DOE and 
the District Government to find ways to provide more robust information about schools and options 
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to families. In addition, WSF will expand its doenmentation of the sehool visits that are performed 
throughout the year. WSF provided the GAO with a list of 42 school visits performed in the 2005- 
2006 school year, including the dates of the visits and the names of the WSF staff attending the 
visits; however, the GAO puzzlingly reported that WSF did not provide evidence of those visits. 

5. Develop procedures to ensure that accurate information is provided to parents before a school is 
chosen about the summary achievement data of students, teacher qualifications, and tuition 
levels, and that schools make such information available on an annual basis to parents of 
enrolled students 

WSF will re-evaluate the methodology by which school information is collected. Schools are 
required to report summary achievement data to families of enrolled students so that families can 
gauge their children’s progress. WSF asserts that this is an appropriate fomm through which to 
provide families with this information. 

WSF, as the administrator of a federal program, has no expertise in evaluating the results of testing 
data, or in evaluating such data as between schools. 

In addition, it requires intimate knowledge of each student to identify a productive school-student 
match. WSF has found that the best such matches are made when families, students, and schools 
work together directly. 

WSF will continue to work with the DOE and the District to provide OSP families access to the 
resources that will assist parents in making the most informed, and best possible, educational choices 
for their children. 

Again, the Washington Scholarship Fund appreciates this opportunity to respond to the GAO’s 
Draft Report. Naturally, we welcome any questions or comments the GAO might have regarding our 
responsive commentary. 

Sincerely, 

Gregory M. Cork 

President and CEO 

The Washington Scholarship Fund 
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